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(57) Abstract 

This invention relates to a system and method for interactive, adaptive and individualized computer assisted instruction. This invention 
includes an agent (108) for each student (101) which adapts to its student, and provides individualized guidance to the student and controls 
to the augmented computer assisted instructional materials. The instructional materials of this invention are augmented to communicate 
the student's performance and the material's pedagogical characteristics to the agent, and to receive control from the agent. Preferably, 
the content of the communication between the agent and the materials conforms to specified interface standards so that the agent acts 
independently of the content of the particular materials. Also preferably, the agent can project using various I/O modalities integrated, 
engaging, life-like display persona(e) appropriate to the preferences of its student, and appear as a virtual tutor to the student. Finally, 
preferably this invention is implemented on computers interconnected by a network. 
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AGENT BASED INSTRUCTION 
SYSTEM AND METHOD 



1. FIELD OF THE INVENTION 

5 This invention relates to a system and method for 

interactive, adaptive, and individualized computer-assisted 
instruction of students, preferably implemented on network 
connected computers. More particularly the system and method 
includes for each student an agent adapted to that student 

1Q which monitors its student's instructional behavior, responds 
to teacher direction, and controls the instructional 
progress, and guides its student, all of which constitute one 
aspect of a virtual tutor. Preferably, the viewable on- 
screen aspect of the agent includes customizable multimedia 

15 presentation personae, which constitute a further aspect of a 
virtual tutor. 



2 . BACKGROUND OF THE INVENTION 

The application of computers in education has been 

2Q limited by several problems, including a failure to provide 
systems that adapt or individualize to each student, a 
failure to integrate systems effectively into the existing 
classroom in elementary and secondary schools, and a failure 
to exploit technological developments. Although during the 

22 last three decades, a number of interactive educational 
techniques have been implemented on computers, all these 
systems lack the ability to recognize and to adapt to each 
student's individual characteristics. The common motivation 
for interactive educational techniques was the recognition 

30 that individual student interaction fosters learning to a 
greater degree than mere passive exposure to a fixed pace 
presentation (Kulik et al., 1986, Effectiveness of computer- 
based adult learning: a meta-analysis, Journal of educational 
computing research 2:235-252; Kulik et al . , 1983, Effects of 

35 computer-based teaching on secondary school students, Journal 
of educational psychology 25:19-26) . Existing, interactive 
educational techniques have many variants: programmed 
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instruction, mastery learning, audio-tutorials, direct 
instruction, personalized system of instruction, precision 
teachmg, fluency learning and others (Engleman et al 1982 
Theory of instruction: Principles and Applications, 
5 Irvmgton, New York; Keller, i 968 , "Goodbye, teacher « 
Journal of Applied Behavior Analysis l:79- 89; Lindsley, ' 1990 
Precision teaching.- By teachers for children. Teaching 
Exceptional Children 22:353-359,- West et al . , 1992 . Designs 
for E X cellence in Education: the Legacy of B.F. Sk inner, 
10 Sorns West, Inc., Longmont, Colorado, pp. 14 7-160) Several 
systems have attempted to harness the interactivity of the 
computer for these interactive educational techniques m 
early work, for example, text-based programmed instruction 
was converted to computer format and implemented on time- 
15 shared systems. This early development was extended with 
more sophisticated computer-assisted instruction <«CAI») 
also known as compute based training (»CBT») . 

In CAI, for example, the computer acts as a teaching 
machine. A program presents instructional displ ays , accepts 
20 student responses, edits and judges those responses, benches 
on the basis of student responses, gives feedback to the 
student, and records and stores the student's progress 
Examples of CAI systems include those of Carbonell, 1970 Al 
m CAI. an artificial intelligence approach to computer-' 
25 assxsted instruction, IEEE Transactions on Man-machine 
Systems, 11:190-202; Osin, i 98 4, CAI on a national scale 
Proc. 4th Jerusalem Conf. on Information Technology, op 4i 8 - 
424; seidel 1971; Koffman et al . , 1975, Artificial 
intelligence and artificial programming in CAI, Artificial 
30 intelligence 6:215-234. Effective CAI instructional 

materials for a limited number of specific topics have been 
developed, as have special "authoring languages," which 
assist instructional developers on the tasks of designing 
instructional materials. U.S. Patent No. 5,310,34 9 is 
3 5 exemplary of such CAI systems. 

However, existing CAI systems do not adapt to their 
students. These systems merely sequence students through 



- 2 - 



WO 97/44767 



PCT/US97/08687 



educational materials, based only on student performance 
during a current lesson and using only parameters such as 
recent responses and pre-requisite patterns. These systems 
do not gather or use information on more comprehensive 
5 student characteristics, such as past student performance, 
student performance on other courses, student learning 
styles, and student interests. 

A greater deficiency is that existing CAI systems do not 
recognize characteristics of their individual students. They 

10 cannot be individualized or made responsive to their students 
styles. Thereby, these system ignore those roles of a human 
tutor that can be of unparalleled significance in the 
education of an individual. The human tutor assists in 
scheduling and prioritizing and in maintaining interest 

15 through proper reinforcement and knowledge of student 
abilities and preferences. A human tutor observes and 
addresses patterns of errors and maintains a consistent 
manner of interaction across a broad range of subject matters 
and activities. Moreover, a human tutor effectively 

20 integrates the cognitive, personal and social aspects of the 
instructional situation. In other words the human tutor 
provides a level of individualization based on student styles 
and on requirements of the instructional task. Furthermore, 
the human tutor provides an equally effective interaction 

25 with the teacher by accepting individualized instructions, 
collecting data and providing detailed reports. By failing 
to address these higher order services and roles of an 
effective human tutor, existing CAI systems fail to fully 
engage their students and thus fail instruct as well as 

3 0 possible. 

Additionally, a further problem of computer assisted 
instruction, particularly in primary and secondary school 
settings is poor integration into the rest of the school 
curriculum and often poor quality of the educational 
35 materials. in application to elementary and secondary 
schools, two main patterns of instructional computer use 
prevail, which illustrate the compromises prevalent today. 
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In one pattern, the integrated learning system (»ILS») is a 
dedicated installation that is used in schools to teach basic 
strands of reading, mathematics and related topics, spelling 
writing, and other language arts, from grades one to six, or 
5 perhaps to eight or nine (EPIE, 1990, Integrated 

instructional Systems: an Evaluation, Educational Products 
Information Exchange, Hampton Bays, N.Y., . The paradox with 
th, s pattern, regardless of the quality of the instruction 

10 li f M ed T thSSe SySt6mS ' W ° rk ° f StUdSntS in 

10 little relation to what goes on in the classrooms in that 

cZor* tOPlC ' ^ fUndamental ~-on is that the teacher 
cannot influence or respond meaningfully to variations in 
student progress or to relations between the CAI curriculum 
and the classroom text, materials and activities. This is 
15 the case even where the ILS installation produces reports in 
some detail as to each student's progress and standing 

standi^ "T* Pa " ern ° f C ° mPUter USe in SCh ° 0ls is that of 
standalone short units on specific topics (TESS 1996 

Database of Educational Software Systems, Educational' 
20 Products Information Exchange, Hampton Bays, n.y ) TheS e 
can be hypercard programs, simulations, or games, and are 
usually separate from the basic classroom curriculum. Though 
occasionally of excellent quality, the paradox here is that 
these products are usually chosen for enhancement, possible 
25 optional, and do not account for a major component of school 
related instruction. No records are kept or returned 
regarding student performance. These programs have not made 
a ma;, or contribution to school instruction. 

Finally, computer assisted instruction systems ha— 
30 ignored or underutilized such important developments in" 
computer technology over the past four years as agent -based 
system, client-server systems, and networking systems 
Though now an active field with a wide spectrum of activities 
from research to commercial applications, application of 
35 agent-based systems in educational, instructional and 

homework tasks has not been explored. Software sy Stems for 
intelligent agents have successfully applied in travel 
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arrangements, email management, meeting scheduling, stock 
portfolio management, and gathering information from the 
Internet (Maes, 1994, Agents that reduce work and information 
overload, Communications of the ACM 32: 30-40 ) . In all these 
5 applications, software agents perform tasks on the user's 
behalf, receiving only general instructions from their user 
but then executing detailed tasks with considerable 
independence and initiative. In client - server systems, these 
agents can operate in the client, the server, or both. 

10 Recently, adaptive and personalized agent based systems 

have begun to be developed. Systems with adaptive agents, 
agents which learn from experience, has made gains with new 
techniques continually identified. Adaptive agents have 
permitted new commercially viable adaptive systems 

15 implemented across networks. In these systems, an agent is a 
"go-between, " mediating relations in a manner whose function 
is understood with details being left to the agent itself. 
The agent acts as a "stand-in" for its user, who is thus 
freed from direct manipulation of the network. In 

20 instructional applications, there is an unmet need for an 
agent who serves two users: the school system and the 
individual student. This is the well-known role of the 
teaching assistant /tutor . Maes, 1994, and others have 
extended the metaphor of agent to that of personal assistant, 

25 an agent who learns some important characteristics of its 
user, and adapts its behavior accordingly. Agents can learn 
by a mixture of methods: observation, receiving feedback 
from its user, receiving instructions from the user, and 
consulting other agents concerning "similar problems." To 

30 combine the important properties of competence, trust, and 
intimacy that a personal assistant should have, an agent 
should be in touch with relevant data, represent important 
facts in a reliable manner, and engage with its user in a 
personal and fundamentally sympathetic --at times playful — 

35 manner. Approaches to the creation of agents with personal 
characteristics have begun to be explored. In this work, 
relevant techniques are found in the tradition of film 
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animators who, through the portrayal of emotions, gave their 
characters the illusion of life. 

Moreover, computer assisted instructional systems have 
only haphazardly exploited the potential of client-server 
5 systems and networking technologies. Client-server 

architectures have emerged as the principal architecture of 
distributed computer systems. Client systems, running under 
sophisticated windowing operating systems, can support 
advanced object based software applications, including high 
10 speed graphics, animation and audio output. Servers can 
store gigabytes of data and programs at central or 
distributed locations at quite reasonable cost Objec* 
oriented database systems have been developed to store" 
structured data on servers. 
15 Client systems, in a striking change from only several 

years ago, now virtually all have multimedia capabilities 
including high quality graphics, sound, and at least limited 
video playback capability. Text- to- speech software is 
presently available for use with these systems, and speech 
20 recognition software is on brink of widespread commercial 
acceptability on low cost platforms. New authoring tools 
support graphical methods for generation of multimedia 
presentations and computer based instructional materials 
having corresponding sequencing logic. 
25 Clients and servers can be linked remotely with 

increasing convenience and decreasing cost . The Internet has 
emerged as a means of providing an inexpensive means of 
connecting computers to provide effective communications and 
access to information and other resources such as software 
30 Further Internet developments that made the Internet truly' 
universal include the HTML and the HTTP protocols, which 
provide platform independent access to hyperlinked multimedia 
information, and the Java™ programming language, which 
provides platform independent software for Internet 
35 applications programming. Subsets of the Internet -- 

intranets -- have become an increasingly important means for 
disseminating information and enabling communication within 
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constrained domains, such as a single school system or 
corporate enterprise. 

Existing CAI systems have not addressed these functional 
deficiencies nor have they exploited the possibilities of 
5 existing technologies. 

Citation of references hereinabove shall not be 
construed as an admission that such a reference is prior art 
to the present invention. 

10 3 . SUMMARY OF THE INVENTION 

The Agent Based Instruction ("ABI") system of this 
invention is a system and method for interactive, adaptive, 
and individualized computer-assisted instruction and 
homework, preferably implemented on network connected 

15 computers, that overcomes these problems by providing the 
following objects in preferred and alternative embodiments. 
This invention provides a more effective system responsive to 
the needs of several parties interested in education. 

An important object of this invention is to provida the 

20 student with a virtual tutor, by having agent software 

("agent") adapted to each student that offers a high quality 
of individualized student interaction and that manages or 
controls instruction in a manner approximating a real tutor. 
The agent exercises management or control over the computer- 

25 assisted instruction materials and provides information and 
help to the student, both synchronously and asynchronously to 
particular instructional materials. Agent behaviors are 
sensitive to both the educational context and to the history 
of student behavior. 

30 In a preferred embodiment of this invention the agent 

integrates data from several sources. From computer-assisted 
instructional materials, it accepts data on the methods of 
instruction adopted by particular materials and on student 
performance in the instruction. From the student, it accepts 

35 direct interactions as well as using the history of previous 
student performance stored in a student data object. From 
the teacher, it accepts data on customization and student 



WO 97/44767 



PCTYUS97/08687 



assignments. From the school, it accepts data on assigned 
courses, data on analysis of student body performance, and 
educational standards and criteria, m a preferred 
embodiment, these inputs allow individualization of agent 
5 interaction. Alternative embodiments are responsive to 
additional data types and sources. 

In a preferred embodiment of this invention, diverse 
agent behaviors are handled uniformly by a single means The 
diverse behaviors include encouragement and feedback 
10 providing meta- cognitive help on ongoing instruction 

managing or controlling and individualizing computer based 
instruction to the student's learning modes, and assistance 
with assignment management . These diverse behaviors are 
selected from a set of potentially appropriate candidate 
15 behaviors. This set of candidate behaviors is ordered and 
the highest ranked behaviors are chosen. 

In a preferred embodiment of this invention the diverse 
agent behaviors adapt to the student based on a variety of 
information about the student. The agent modifies its 
20 behavior on the basis of a growing history of interactions 
with the student over time, as this history of student 
performance is stored in the student data object. The agent 
can also modify its behavior on the basis of teacher and 
school system supplied information. 
25 Another important object of this invention is that the 

agent presents itself on-screen to the student with 
integrated, and optionally, animated multimedia persona or 
preferably a plurality of persona (hereinafter called 
"personae"). The on-screen agent can appear as living 
3 0 entities, which in grade school can be comfortable "Study 

Buddies- and in adult training can be appear as an objective 
"Concept Coach". The on-screen agent instructs, motivates 
engages and guides its student. 

In a preferred embodiment, the on-screen agent can be 
35 dramatized by a single character or by a cast of interacting 
characters. The interaction between these actors can be 
individualized to reflect the pedagogical response of the 
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agent. To maintain student interest in the agent response, 
story lines continuing across materials or session can be 
used . 

In a preferred embodiment of this invention the voices, 
5 gestures and motions of the personae are derived from the 
chosen behaviors, student personae preferences, and the 
history of recent behavior by selection from tables 
containing a rich variety of alternative sound and visual 
display objects. All elements of the on-screen agent display 

10 are then synthesized in an integrated display script calling 
for graphics, animation, video, or sound as appropriate. 
These scripts are then bundled into applets, run- time program 
fragments that represent a complete element of performance. 
This display is highly configurable by the student, the 

15 teacher, or the system administrator. 

It is an advantage of this invention that elements of 
the display objects can be created by artists, animators, 
singers, and so forth, as data snips. Pluralities of data 
snips can be stored in libraries of dynamic clip art and then 

20 installed in an implementation of this invention. In this 
manner the on-screen agent personae have an appropriately 
contemporary, realistic, and engaging manner. Data snips 
are, in general, short clips of sound, voice, graphics, 
animation or video, or combinations of these used to 

25 construct the on-screen agent. A data snip can also be a 
complete pref ormatted animated sequence, perhaps in the 
format of a talking animated daily cartoon strip. 

Another important object is that the method and system 
cf this invention is adapted to implementation on a variety 

3 0 of networks. When so implemented, the interactive, adaptive, 
and self-paced computer-assisted instruction and homework 
provided by this invention is available to geographically 
dispersed students and from geographically dispersed schools. 
For example, an implementation of this invention as a 

35 "Homework Network™" can make computer assisted homework 

available to students of all levels at home. In addition to 
computers located at residences of students, the student can 

- 9 - 
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also access homework materials at computers located in youth 
centers, libraries, schools and other locations. 

In a preferred embodiment, the network on which this 
invention is implemented as an intranet configured of 
5 appropriate links and utilizing the known TCP/IP protocol 
suite, and as appropriate, ATM technologies, including World 
Wide Web. associated browsers, and mail format extensions 
Implementation over the public internet is equally preferred 
xn cases where extensive connectivity is needed. 
10 A further important object of this invention is to 

utilize augmented computer-assisted instruction materials 
which present to students a variety of interactive, adaptive 
and self-paced computer-assisted instruction and homework 
materials in a manner which informs the agent of a student's 
15 progress and performance and which permits the agent to 
manage or control the materials to the student's pedagogic 
characteristics. Thereby, the ABI system can effectively 
guide and engage students in their educational tasks. 

In a preferred embodiment, these instructional and 
20 homework materials are composed of materials data presented 
by a materials engine. The materials data includes display 
objects containing the substance of the instruction, logic to 
sequence the display according to student input, and 
notations. Notations are augmented definitions that serve to 
25 pass information to the agent concerning the materials and 
the student. For example, notations classify key sections of 
materials which are educationally significant student 
actions. Preferably, authoring tools assist in developing 
these augmented instructional materials. Materials tasks and 
30 sequences are created and entered by instructional designers 
and subject experts. Notations are usually entered by 
instructional designers and can be customized by teachers. 

In a preferred embodiment, the information passed in the 
notations is standardized according to an instruction 
35 materials interface standard. This standard establishes a 
uniform way the materials independent data relating to 
student performance are to be provided to the agent and a 
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uniform way for the agent to guide the student in a materials 
independent manner. 

A further important object of this invention is to 
provide to the student a range of tools which are integrated 
5 with the agent in a manner similar to the instructional 
materials. These tools include general tools helpful to 
assigned instructional tasks, and special tools for group 
work and communication and for student scheduling. 

In a preferred embodiment, the general tools include at 

10 least a calculator, an encyclopedia, a dictionary, a 
thesaurus, each appropriate to the several levels of 
students, which can access an ABI implementation. In a 
preferred embodiment, the group work and communication 
materials allow, when permitted, message exchange, student 

15 linking into groups for joint work, and student linking into 
groups for structured work such as contests. In a preferred 
embodiment the student scheduling tool records assigned 
student activities and their priorities. In an embodiment, 
this tool can be consulted by the student to view schedules. 

20 It can be consulted by the system to prescriptively schedule 
required activities, to permit student choice, or. to permit a 
mixed scheduling initiative. Finally, it can be consulted by 
the agent to offer scheduling advice to the student. 
Typically, student assignments are set by a teacher. 

25 An object of this invention is reporting of student 

performance to students, teachers, parents, administration, 
or to other appropriate individuals in a business enterprise 
or other commercial versions. These reports include the 
unique data on the student's pedagogic performance 

30 accumulated and analyzed by the agent, as well as all the 
usui.l and expected performance data on specific materials 
available in existing computer-assisted instruction systems. 
In a preferred embodiment this data is derived from the 
student data object, where all permanent student data is 

35 stored. These data objects are preferably stored in an 
object oriented database system against which are run 
reports of this data. It is an advantage of this invention 
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data on h t 0 r COn H teXt ^ haVe access " curr«t 

in th drSn ' ^ thSreby Play a — i»"««d role 

in their children's education. 

° b ^t of the invention is to utilize current 
5 technology for student interaction. when available this 
invention is adaptable to Network Computers <-«;., NCs - re 

orthToubr^r" SP6CifiCally access 'intranets 

and Ll t6rnet - ^ 3 Pref6rred ^odiment 

and implementation, this invention is adaptable to multimedia 
10 km tor some students, and to such special interaction 

tudl s ° 91e ; h aS ^ adVanta * eous to special students or 
students wxth special needs. Typical interactive devices 
xnci ude keyboardS( mice or other pQinting ^ xc.. 

15 21 I 1 "' j ° y - StiCkS ' tOUCh "tivated devices, light-pens 
15 and so forth, other devices, such as virtual reality 
devices, can be added as they become commercialized 
It is clear to those of skill in the arc chat 

lZT7 inteZaCtiVe ' adaPtiV6 ' ^ self 'P-ed computer- 
assisted instruction and homework delivered over widely 
available computer networks this invention has immediate 
application in public, private, and commercial school 
environment of all levels. Educational research shows that 
instruction and homework of these characteristics improves 

25 lllT™ 5 ' 6dUCati0nal ° UtC ° meS - F -ther, in school contexts 
25 this invention advantageously provides immediate access to 

student performance and pedagogic characteristics to all 

interested parties, including parents. 

4 - PRIEF DESCRIPTIONS n P the npawraqc 

30 These and other objects, features, and advantages of the 

invention will become apparent to those of skill in the art 
m view of the accompanying drawings, detailed description 
and appended claims, where: 

Pig. 1 illustrates in overview fashion the princioal 
35 functional components of and parties in the ABI system' 
Figs. 2A and 2B illustrate in overview fashion an' 
implementation of the functional components of Fig i . 
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Fig. 3 illustrates an exemplary student display screen 
of the implementation of Fig. 2; 

Fig. 4 illustrates in more detail exemplary screen 
interaction between the on-screen agent and the instructional 
5 materials of the implementation of Fig. 2; 

Fig. 5 illustrates an exemplary interaction of a student 
with the ABI system implementation of Fig. 2; 

Fig. 6 illustrates in more detail the software 
components and hierarchy in the implementation of Fig. 2; 
10 Fig. 7 illustrates exemplary message flow through the 

implementation of Fig. 2; 

Fig. 8 illustrates agent action processing of Fig. 7 in 
more detail; 

Fig. 9 illustrates agent behavior processing of Fig. 7 
15 in more detail; 

Figs. 10A and 10B illustrate the structure of student 
data object of Fig. 7 in more detail; 

Fig. 11 illustrates exemplary processing of the student 
data object of Fig. 7; 
20 Fig. 12 iilustates an exemplary sequence of student 

metarequests and agent metaresponses . 

5 . DETAILED DESCRIPTION OF THE INVENTION 

Sec. 5.1 presents a general overview of the Agent Based 
25 Instruction system. Sec. 5.2 describes the preferred 

hardware and operating software configurations. Sec. 5.3 
describes details of the instructional interface between the 
AEI system and its users. Sec. 5.4 describes in a general 
fashion the software structure of the ABI system with 
30 subsequent sections describing each component in a more 
detailed fashion. Sec. 5.5 describes the instructional 
materials and the tools in a more detailed fashion and Sec. 
5.6 describes the agent in a more detailed fashion. Sec 5.6 
includes detailed description of the preferred interface 
35 between the agent and the materials in the ABI system. 
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5mlm ABI Svflfr«*m Qyprv^^ 
rn the following, the systems and methods o£ chis 
xnventxon are described in the context of a school system 
with examples drawn privily £rom e i ementary 

of skill ln the relevant arCE that th . s invent . 
applied at all levels of public and private education Ln 
pre-scho=l through university, and to al! forms o, colercL 
or corporate education. ln a n these ™^ 

ZTnT haS 1 P ""= Ul " in making education and 

school T " SCh ° 01 ' " the " >«»■. « 

schools with geographically dispersed students and to 

students at geographically dispersed schools, and at other 
types of locations. Further, it win be apparent that this 
15 invents can be applied in contexts . other than education 
where mon.tored interactivity and individualization a I" be 
provrded, as in child care or weight loss 

The following glossary contains several terms frequently 
used in the Detailed Description of the invention. I T 

rmoT" " ™ aid " ort " " ^™duc 

™° re fUUy deSCribSd "* 

m the following sections. 

Agent: agent software together with the data it references 
25 executing in an ABI system. 

Agent Based Instruction ( "ABI " ) System: the novel 

instructional system of this invention, preferably 
comprising an agent for responsive, adaptive and 
ind vitalized instruction of students and a networK for 
d s r lbuti on of instruction, which projects the personae 
of the virtual tutor. 



35 



Agent Software: software modules that generate responsive 

adaptive, and individualized behavior in the ABI S y St e TO 
preferably implemented according to methods from 
artificial intelligence. 
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Applet: an executable program fragment advantageously 

downloaded to a client across the network, in the ABI 
system applets are particularly used to represent a 
complete element of on-screen agent actions, or 
5 performance, {e.g., a character scratching its head and 

saying an utterance) and can reference various data 
snips of animation, sound, and scripting information. 

Authoring Tools: programs used by instructional designers to 
10 develop materials data, such development includes 

inserting notations. 

Cast: a plurality of persona ("personae") representing the 
on-screen agent. 

15 

Character: an individual persona in the cast of the on- 
screen agent . 

Concept Coach: a possible alternative name for a persona in 
20 the cast of an on-screen agent that is suitable for high 

school and adult students. 

Data Snip: an elementary piece of sound, voice, animation, 
video, or a graphic; data snips can be combined, 
25 preferably by an applet, to represent a complete element 

of on-screen agent action. 

Electronic Learning Friend ("ELF") : a generic name for any 
persona in the cast of an on-screen agent; also a 
30 possible alternative name for a persona suitable for 

middle school students . 

Instructional Materials: the components of a course of 

instruction, such components are selected according to 
35 the course and can include prerequisite tests, pre- 

tests, lessons, and unit tests. 
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Materials Data: the content Qf instructional 

Materials Engine: software modules that reference 

instructional materials data and tools data to present 
the instruction and the tools to the student. 

Meta revest: . student request direcay ^ ^ ^ 

agent, an exemplary request is 'asking for a hint.' 

10 Meta-response: all responses to a student produced by the 
agent software, as distinguished from presentation by 
instructional materials, tools and communications. 

Network: the hardware and software connecting student client 
computers to school servers in an ABI system,- the 
network connections can comprise fiber optic links, 
wired links, or terrestrial or sate l lice wireless links. 

2 o Notat da°:: : th a r erface info ™ tion ^ « t .n.i. 

data that causes the materials engine to send and 
receive messages from the agent software,- preferably 
notations create standardized interface messages between 
the agent and the materials . between 

25 On-screen Agent: presentation by the agent software on the 
student's display using such media as sound, voice 
graphics, animation, video, or other multimedia ' 
modality; this presentation preferably displays one or 
more life-like personae 

30 

Persona: a character in the cast of an on-screen agent. 
Personae: the collective plural of persona. 

35 Student Data Object: data about each student which the agent 
software references in order to provide responsive 
adaptive, and individualized instruction to that ' 
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student; this data is updated during course of each 
lesson and is advantageously stored as one object, or 
alternatively a few linked objects, in the ABI system. 

5 Study Buddies™: a possible alternative name for personae in 
the cast of an on-screen agent that is suitable for 
elementary school students. 

Tools Data: the content of tools supporting particular 
10 instructional materials; tools can include a dictionary 

a calculator, or an encyclopedia; and so forth, and 
tools data are the content of the dictionary, the 
calculator, or the encyclopedia. 

15 Utterance: a text or voiced response by on-screen agent. 

Virtual Tutor: the ABI system components acting together to 
emulate a human tutor; from an individual student's 
perspective, the Study Buddies™, ELF, or Concept Coach 
20 appears as his or her personal tutor. 

The following sections of the Detailed Description elaborate 
each of these terms and describe their connection and 
interaction so that the ABI system presents responsive, 
25 adaptive, and individualized instruction and assumes the form 
of a virtual tutor. 



5.1,1. Functional Components 

In view of these objects and advantages, Fig. l 
30 illustrates the principal actors and the principal functional 
components in an ABI System. These include, generally, 
materials engine 102, agent software 108, and student data 
object 109, all of which interact with student 101 and with 
teachers and administrators 106 via a computer network 
35 described below in conjunction with Fig. 2 to create a 

virtual tutor of student 101. Student 101 is typically one 
of many students enrolled in a school or similar institution. 
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to each r T ~ " tUt ° r ^^-d 

to each student, which formed by the functioning of agent 

software 108 with student data object 109, which stores 

5 leTZTT 3 ° f StUdenC ^ assi *™^ — standards 
set by teachers and administrators 106. other actors not 
shown in Fig. 1 can be relevant ^ particular applicat 

for example, parents in the case of primary and secondary 
education. y 

10 • Mat6rialS en9ine 102 P resent - educational content such 
10 as mstructxonal units, homework assignments, and testing to 
student 101. The educational content includes instructional 
«t.rx.l. data in, communications materials data 104 . Z 
tools data us. instructional materials data i 14 include 
15 TT inStrUCti ° nal -t.ri.la similar to those known 

15 he art but, importantly, augmented with notations for use 
in thx. invention. The materials also include various tools 
115 appropriate to particular instructional materials, such 
as a calculator for mathematics, a dictionary f or writing 
access to on-line reference sources for research, and so ' 
20 .orth. Further, materials can also include communication 
materials data 104, which define and provide communication 
with other students 105 for instructional purposes. Such 
purposes can provide, for example, the tutoring of one 
student by another more advanced student, joint work of 
25 several students on one instructional materials lesson as 
permitted, and educational contests, such as spelling bees 
Further, this invention is equally adaptable to other forms 
of materials that function in the framework of Fig j. in 
particular interfacing to the agent software as indicated by 
30 arrow m, and that are useful in a particular appl ication of 
this invention. For example, materials appropriate for child 
care contexts can differ from the above three classes by 
perhaps, having different paradigms of interactivity 

The structure and course of interactions 103 between the 

oTeZll 7 mat6rialS ^ Pref6rably * 

of educational psychology and sound educational practice 

such as are described in the exemplary reference Englemann et 
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al " 1982 ' Theory of instruct ion: p ri nciples and 
applicfrtipns, New York: Irvington Publisher. At the most 
immediate level, for example, during homework or instruction, 
student 101 can make requests and receives responses from 
5 materials engine 102 and, in turn, materials engine 102 can 
make requests and receive responses from student 101. The * 
materials engine can adjust its sequence of presentation in 
response to student responses. At a next level, the requests 
and responses exchanged between the student and the materials 
10 engine can follow several patterns known in the arts of 
computer based instruction and which, for example, include 
the following. First, the student can respond to questions 
presented by the materials engine, and in the course of 
responding, can ask for advice or hints, the use of a tool 
15 such as a calculator, or other relevant assistance. Second, 
the student can advance to the next item, lesson, or unit 
upon successful completion of the present item, lesson, or 
unit. Third, in case of error, the student can request, or 
automatically be presented with, appropriate repeat, review, 
20 or remediation materials. Finally, at a higher level these 
patterns of interactions can be analyzed to provide more 
adaptive responses from the system. 

Teachers and administrators 106 also interact with 
materials engine 102 for several purposes, as represented by 
25 arrow 107. For example, teachers can customize existing 
materials by adding additional items, modifying existing 
items, altering the order of item presentation, changing the 
notations (see infra.) governing agent interaction, and so 
forth. A teacher can create particular instances of 
30 materials suitable for one class, a group, or even one 
student. Further, although reporting of student progress 
preferably occurs by other means, in alternative embodiments 
the materials engine can directly report student progress to 
teachers and administrators. For example, this can be done 
35 by entering notations that generate messages for the 
teachers . 
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Also, instructional designers can create, or "author, • 
materials for use in this invention. Such materials can be 
original or can be derived from existing textbooks, workbooks 
or media material. They can employ standardized curricula, 
5 pretests such as criterion tests, post -tests, and 

standardized texts. Authoring instructional materials in a 
course suitable for interactive instruction typically 
comprises several steps, including decisions about the 
objects to display to the student, the sequencing of these 
10 objects, and the interactions with the agent. The £i££l step 
is the selection of objects which carry the education content 
for presentation to a student. Objects can include visual 
display items, such as text, graphics, animation or movies, 
audible display items, such as voice, audio and so forth. 
15 They can include input items known in the computer arts, such 
as buttons to select, selections to chose from, text to 
enter, hypertext and hypermedia links, functions to perform 
with student input, and so forth. The second step is the 
selection of the sequencing logic for the ordered display of 
20 the objects to the student and the educationally appropriate 
reaction to student requests and responses. The sequencing 
logic can reference instructional controls set by agent 
software 108, such as a command to increase example density, 
and preferably is chosen in light of principles of 
25 educational psychology and practice as detailed above. The 
third step is the specification of interactions with a 
student's agent or virtual tutor, a key component of the ABI 
system. This specification is made by augmenting the 
sequencing logic with "notations," which are referenced, 
30 called, or executed by the sequencing logic during object 
presentation and that communicate with the agent, in a 
preferred embodiment by exchanging messages. In the ABI 
system, the agent builds an adaptive model of its student's 
pedagogic characteristics, in other words the student's 
35 cognitive styles, by monitoring the course of the student's 
interactive instruction. The notations are the means for 
this monitoring. Finally, in order to make materials 
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available, the authored materials are indexed and stored in 
the files of the ABI system, preferably on materials server 
systems . 

ABI authoring tools differ from authoring conventional 
5 instructional materials in that notations are present in 
these materials to enable the agent software to update the 
student data object, to monitor and modify the instruction, 
student's use of a tool, or a communication task. ABI 
authoring tools support and facilitate the conversion of 

10 existing materials to the ABI instructional format. 

As indicated above, an agent unique to each student 101 
is important in the ABI system. A student's agent is 
comprised of agent software 108 in conjunction with a student 
data object 109 unique to each student. As the agent 

15 software monitors its student's instruction, it builds an 
adaptive model of its student in student data object 109. 
Guided by this model, agent software 108 acts, first, to 
manage or control the student's instruction, and second, to 
directly guide the student in order that the total ABI system 

2 0 can present education to each student in an optimal fashion 
best adapted to the student's evolving abilities, skills, and 
preferences. In other words, the agent becomes a virtual 
tutor by acting as a student's personal and individualized 
tutor. First, the agent manages or controls instruction of 

25 student 101 by directly controlling materials engine 102 in 
its presentation of materials data 104, 114, and 115 through 
interaction with the materials engine, as represented by 
arrow ill. The agent preferably manages or controls the 
materials engine in two manners, synchronous with materials 

30 data presentation, such as when the materials engine 

encounters an appropriate notation in the data, makes an 
agent request, and waits for an agent response, and 
asynchronous with the presentation, such as when the agent 
software adjusts control parameters which the materials 

35 engine references at decision points. Examples of 

synchronous control are an" instructional material asking the 
agent for permission to allow the student to use a tool, to 
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receive a hint, or to be given remediation, or a 
communications material asking the agent for permission to 
permit the student to engage in a particular type of 
communication with certain other students. An example of 
5 asynchronous control is the agent setting of pedagogic 
parameters, such as coaching parameters that the materials 
engine uses to adjust its presentation, according to the 
pedagogic characteristics of the student. Exemplary coaching 
parameters include the time pacing of exercises, the new 
10 concept seeding rate and the density of examples, m this 
manner, the materials can present interactive instruction 
according to optimal values of the pedagogic characteristics 
or cognitive styles of each student as determined from the 
agent's observation of its student. 
15 Second, agent software 108 directly guides the student 

by exchanging communication with the student, as represented 
by arrow 112. Student communication also preferably occurs 
in two manners, synchronously, in which the student directly 
makes meta-reguests of the agent tutor and receives meta- 
20 responses and second, asynchronously, in which the agent 
tutor itself generates a meta-response in response to some 
instructional event. Herein, requests and responses are 
prefixed with "meta" when they are exchanged directly with 
the agent. Meta-requests include student questions to the 
25 agent - for example: How am I doing? What should I do next' 
Could you say that another way? - or student requests - for 
example: I need a hint; I need help. The agent responds to 
each student question or request. Agent meta- responses can 
be generated, for example, when the student takes too long to 
3 0 complete an exercise, when the student makes a series of 
repeated errors, or when the student achieves good 
performance. Agent meta -responses can be drawn from such 
categories as reminders, encouragements, reinforcements, 
paraphrases, jokes, progress summaries, and so forth. 
35 Communication with the agent, represented by arrow 112, 

include direct student meta-requests that generate agent 
meta-responses. Other communications derive from 
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instructional event messages generated and communicated by 
augmented notations in materials 104, 114 , and 115. An event 
received by the agent can generate no meta-response at all or 
alternatively can generate an asynchronous type agent meta- 
5 response. At educationally significant points, as the 
materials sequencing logic presents display objects to the 
student and receives inputs from the student, the materials 
data author places one or more notations. When these 
notation are referenced, called, or executed, important 
10 variables and parameters educationally relevant at this 

significant point are gathered into a message, along' with an 
indication of the type of the educational event. These 
messages are events which are then sent to the agent . For 
example, an educationally significant point is the beginning 
15 of a new instructional sequence. The corresponding event 

message can include an indication of the topic to be covered, 
the expected level of difficulty, the expected time to 
complete, and the educational paradigm adopted. Another 
educationally significant point is the receipt of a wrong 
20 answer. in response, the materials can generate several 
messages: a first message can include the time required to 
make the answer, an indication of the type of error, and an 
indication of whether the answer is completely wrong or only 
a near miss; a second message can include text parameters 
25 ("say-it" type message) if the agent chooses to make a 

specific text or spoken comment about the error; finally, a 
third message can include the screen location best 
representing the error ( "point -it » type message) to use if 
the on-screen agent chooses to point to the error or move to 
30 the location of the error. Another educationally significant 
point can be a long delay in receiving the next student 
input, at which point the materials engine can send an 
asynchronous message indicating the time elapsed. Tools data 
115 generate events similar to messages from instructional 
35 materials. Communications materials 104 can generate events 
recording a communication request or an initiation of 
communication with certain other students for a certain task. 
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in the case of shared work on one materials, communications 
materxals can generate events recording how this student in 
progressing with the shared materials; in the case of a 
contest such as a spelling bee. events recording how this 
5 student is progressing in the contest with respect to other 
contestants . 

in addition, in a preferred embodiment agent software 
108 also receives messages describing the progress of the 
student through specific instructional materials For 
10 example in the case of mathematics materials, such messages 
can include information that the student is making errors in 
problems requiring finding common denominators. These event 
message should preferably all information that can be of 
interest to teachers and administrators for tracking student 
15 progress and tracking course adequacy. 

It is important that communication between the student 
and the agent be engaging. Agent communication preferably 
util izes all the modalities of input and output available in 
a particular implementation of this invention, including 
20 text, audio displays such as voice and sound, and video 
displays such as graphics, animation, and realistic movie 
clips. For example, in the case of a communication triggered 
by good performance, the agent can select the display of 
sound and video clips, from a data snips library, that the 
25 student finds pleasing. The agent can further make reward 
graph.cs available on the student's screen for a period of 
time, on the other hand, in the case of error the agent can 
point to the screen location of the error. 

Further, it is highly preferable that the on-screen 
30 agent can assume various display personae during student 

communication. Herein, persona means the effect conveyed to 
the student of the combined and coherent presentation of 
multiple display modalities to emulate a particular 
apparently living, personality. For example, in the case of 
35 elementary education, this can be the selection of tone and 
animation to emulate a pleasant animal or a known cartoon 
character. In some cases, characteristics of the display 
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persona can be selectable by the student according to the 
student's preferences. In other cases, the personae can be 
specified by the instructional materials, the teacher or the 
administrator overriding student persona preferences. 
5 Personae for an elementary school student can be selected 
from well-known cartoon characters and can perhaps be called 
"Study Buddies™. " Persona for commercial or corporate 
education can be adapted to the organizational ethos and can 
perhaps be called a "Concept Coach." Presentations for 

10 intermediate levels can be called an Electronic Learning 
Friend ("ELF") . Even though the ABI system through its 
network, software and database acts as the student's virtual 
tutor, from the elementary school students point of view, the 
"Study Buddies™" are his/her personal tutor. Realism in 

15 voice, gestures and movement reinforce this relationship. 

Agent software 108 in the ABI system builds an adapting 
pedagogic or cognitive model of its student in student data 
object 109 that is independent of the specific materials. 
Event messages to the agent software from the materials 

20 engine preferably include the information from which this 

model is built. In general, event messages must include such 
content as is necessary to describe and parametrize the 
pedagogic or cognitive style models adopted by the materials 
in an implementation of the ABI system. 

25 In an preferred embodiment, the student data object 109 

collects all the permanent data about the student maintained 
by the ABI system. The data objects for all the students are 
collected for permanent storage in a database system. 
Preferably, this is an object oriented database, although 

3 0 this data can be advantageously stored in standard relational 
databases. In an alternative embodiment, however, the 
various subtypes of student data in the student data object 
can be separated into separate objects and stored in separate 
databases. In particular, it is possible to store the 

35 materials specific progress data separately from the 

materials independent global student data. Preferably, the 
student data object is stored as one structured object. 
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Alternatively it can be stored as a plurality of objects, 
each object of the plurality perhaps storing only one subtype 
of data. y * 

The student data objects are accessed not only by the 
5 agent software 108. but also by teachers and administrators 
106 as depicted by arrow 110 . the data object is referenced 
by the agent in order to generate its actions and is updated 
by the agent as it processes events and student meta- 
requests. As depicted by arrow 113, the data object is 

10 referenced by teachers and administrators in order to track 
the student progress and to generate reports concerning the 
students and materials in the ABI system. Teachers also 
update the data object to enter schedule information for the 
student's assignments. Administrators update the object in 

15 order to enter the courses and materials the student must 
master and specify standards and criteria the student must 
meet . 



5 - 1,2 - Exemplary Inmlgmon^.nq Sfcnirn,-^ 

20 Fig S . 2A and 2B illustrate an exemplary preferred 

structure implementing the principal conceptual and 
functional components of the ABI system as illustrated in 
Fi9. 1. A preferred implementation of this invention is 
based on a plurality of computers interconnected bv a 
25 network. Preferably, although all system functions can be 
performed on all interconnected computers, certain computers 
are specialized for certain functions, such as student client 
systems for providing student access or system servers for 
storing system software resources. Therefore, an exemplary 
30 preferred ABI system includes one or more student client 
systems 201, at which student 202 receives instructional 
presentations including homework, other student clients are 
generally indicated at 203 and can be located at school at 
home, or at the office. The system further includes one or 
35 more servers 204, at which teachers and administrators 205 
gam access to the system. Alternatively, they can access 
the ABI system at their own client computer systems 
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These components are interconnected by a network which 
consists of transmission medium 206 and local attachments 
207. Although the network illustrated in Fig. 2A is of a 
bus -type configuration, as in a local area network perhaps 
5 using collision detection or token passing protocols or both, 
this invention is adaptable to all forms of networks which 
support adequate transmission protocols, such as those that 
are functionally similar to the TCP/IP protocol suite, and as 
appropriate, ATM technology. Thus, the invention is 

10 adaptable to networks constructed from switched or non- 
switched links to a central server, which can be configured 
of several LAN attached server systems, to networks including 
CATV cable or optical links, to networks including radio 
links either terrestrial or satellite, and to public or 

15 private packet switching networks. 

A preferred public packet switching network to which 
this invention is adaptable is the Internet. ABI system 
nodes can be linked to the Internet in any convenient manner 
known to those skilled in the art and can include security 

20 boundaries (known a firewalls) to prevent unauthorized 

access. Alternatively, an ABI system implementation can be 
an "intranet," that is a network constructed according to 
TCP/IP protocols but using private links of the types 
enumerated above, and as appropriate, ATM technologies. 

25 In more detail, student client system 201 includes 

memory 208, which is RAM type real memory or a virtual memory 
based on RAM type memory and a backing store. Preferably, to 
reduce size and cost, the client system has no permanent disk 
storage. When available, a preferable student client can be 

30 a low cost network computer ("NC") . A NC is a computer with 
processor, RAM, and network interfaces sufficient to access 
intranets or the Internet. A NC is preferable in cases when 
high-bandwidth network access is available. In cases of low- 
bandwidth network access, the client system can have one or 

35 more disc drives 209 which can be used as a pre- fetch buffer 
or a read-only cache. The" disks preferably are magnetic, 
although in a less preferable embodiment they can also 

- 27 - 



WO 97/44767 

PCTYUS97/08687 



include CDROMs. This optional capability serves to enhance 
communications efficiency in cases where the network has 
relatively low bandwidth. Large files can be downloaded in 
advance of a student session or the student client can cache 
5 read-only data across sessions obviating the need for 

downloading such files. Such caching requires the operating 
system components to maintain some form of version control of 
the read-only data. in any case, the student data object 
which contains all permanent and read-write student data ' is 
10 stored between sessions on a server. This permits a student 
to access the AB1 system services from any available' client 
system at any time by simply downloading the student data 
object to that client system. 

The student interacts with a client system using any 
15 appropriate interactive input/output ("I/O") modes 210 For 
input, standard devices include pointing devices, such as 
mouse 211 or a trackball, age appropriate keyboards, 
optionally speech recognition, and so forth Speech 
recognition will permit brief conversations with the "Study 
20 Buddies™," or other personae, in limited areas The 

invention. is adaptable to special input devices appropriate 
for particular groups, such as the handicapped, and to 
devices yet to be constructed. Virtual reality f'VR") 
interface devices such as VR gloves and VR display helmets 
25 can have immediate applications for special needs students 
For output, preferable devices include computer display 212 
for displaying objects such as text, graphics, animation, and 
video, and audio output devices for voice and sound clips 
The audio and voice can be constructed from data snips stored 
30 as digitized sound files in libraries. Alternatively, voice 
can be synthesized from text. The invention is also 
adaptable to special output devices for special classes of 
students, e.g. the handicapped, and to new devices being 
developed. 

35 Fig. 2A also shows an exemplary screen layout for a 

student client that exemplifies the principal functions of 
this invention. The screen is preferably partitioned so that 
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principal components of this invention are displayed; and 
important student actions are represented by icons or 
buttons. Thus, the screen includes materials and tools area 
220 to the left, agent area 215 to the right, and a system 
5 toolbar 218, which includes a student customization area 221 
at the top. The size of the v screen partitions illustrated 
preferably change from time to time in response to student 
customization or display requirements. In particular, either 
the materials area or the agent area can enlarge, perhaps up 

10 to the entire screen as needed. Materials area 220 is for 
the instructional materials, tools, and communication 
materials to present visual display objects and for these 
components to receive interactive input. This area is 
further subdivided into display region 213 and a materials 

15 specific toolbar 214. On-screen agent area 215 is for the 
on-screen agent to receive meta-requests and to display 
synchronous and asynchronous meta- responses . Also 
illustrated is an exemplary on-screen agent consisting of a 
single persona 216 and a meta-request icon 217. The persona 

20 can move to other screen areas as required. Both these 
components are illustrated as appropriate for elementary 
education. Other visual appearances would be appropriate in 
other situations. The system area at top includes toolbar 
218 for selecting particular available system components. In 

25 particular, always available on this toolbar are selection 
icons 219 for the calendar and scheduling tool. Part 221 of 
the system area can be reserved for student customization, 
for example for the display of reward graphics "given" to the 
student by the agent or virtual tutor. 

3 0 Functionally illustrated in Fig. 2A is an exemplary 

memory organization of a student client system when a session 
is in progress with materials being presented. Layer 222 
comprises operating software and network communications. 
This software provides, among other services, support for I/O 

35 devices attached to the client, a file system with cache 
control, lower level network protocols, such as TCP/IP and 
ATM, and higher-level network protocols, such as HTTP V2.0. 
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Basxc shared ABI system capabilities are provided by 
executive software 223. The executive software verities 
student identity and access authority, establishes 
communications sessions with the system servers as required 
5 dur^g client start-up, downloads from the student object 
database the student data object corresponding to the student 
xn session at this client system, downloads instructional 
mate rlals scheduled for this student, and download executable 
software reared from the systems servers as-needed. The 
10 instructional material and the software are read-only and are 
not changed. The student model is updated by the agent 

the rl T r SCUdent SeSSi °" "* m ° di£ied «• »P»«id to 

the student database on a server for storage. Such 

downloads can utilt „ higher w „ et>jorJ< 

15 protocols, or alternatively direrriv „«, i 

protocols. Erectly use lower level network 

Agent software 225, certain parts of student data object 

226, and certain instructional materials software 224 have 

already been downloaded. The materials are displaying 

20 ejects in screen area 220, forwarding events to the agent 

and receding agent management or controls, as indicated by 
arro w 7 . The agent is ^ ^ ^ 

I 2 ' ^ ^e materials, as represented by 

25 L7 l \ 13 "^^^ and ^ing d"a in student 

25 data model 226, as represented by arrow 228 

Finally, the student client system further includes 
standard components not shown, such as a microprocessor and 
input/output interfaces. Alternative implementations of the 
tudent client system are within the scope of this invention 
30 in particular, student client system function can reside on a 
server system. Further, the client system can be implemented 
as two machines, wherein a first machine performs 
substantially all the computations and a second machine is 
for student interaction and sends student input and receives 
35 display instructions from the first machine. 

The ABI system further includes one or more server 
systems 204 of Fig. 2B with sufficient large capacity discs 
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230 for storing all student data objects in the student 
database, all instructional materials, and all software used 
in the system. The network is used to distribute the 
software, student data objects and instructional materials 
5 form the servers. In alternative embodiments, there can be 
more than one server with software and data component storage 
divided as convenient across the servers. In a further and 
less preferred embodiment, the server can be a central host 
system. In comparison to the client systems, servers 

10 preferably have increased performance and higher speed 
network connections 231 in order to make this stored data 
quickly available to the one or more student client systems. 
Access to the body of student data allows teachers and 
administrators to track student performance by class, grade, 

15 subject, school and so forth. This statistical data is also 
input into agent processing. 

The server systems are preferably configured as shown in 
Fig. 2B and are loaded with software 232 providing the 
following function. First, there is operating system, 

20 network services, and file server layer 233. In addition to 
the functions of layer 222 in the client system, layer 233 
also provides a file server facility including file backup 
and version control. System manger 234 includes facilities 
for access control, authenticating student access requests 

25 and limiting file access to authorized users. For example, 
students can be limited to only their personal files; parents 
to their children's files and curricular related data; 
teachers to files and student data objects for their classes; 
while certain administrators have unlimited access. The 

30 system manager can also maintain any necessary system logs. 
Student data object database 235 is explicitly illustrated 
Student data objects reside on the server systems when the 
associated student is not in session. These objects contain 
the data which is the source of all teacher and administrator 

35 reports, data by which these staffs schedule student courses 
and assignments, and data representing the pedagogic model of 
the student used by the agent software. Instructional 



WO 97/44767 



PCTYUS97/08687 



T tab " eS ^ o« executable 

software also reside on the server system. When group 
caseations is in use. agents and communication tasks 
monitoring the groups can reside on the servers. The servers 
5 also contain areas 237 tor administrative data and for 

reports and report software o£ interest to f h= ,„ • • 

staff ,. interest to the administrative 

211 / ^ 5 SerVCtS COn " in «eas "8 for 

data and report software of interest to teachers 

10 availa^T"' T*" can be made 

available specif ically for the teaching staff, along with an 
individuated teacher agent acting as a virtual tutor or 
the teachers. The facilities of the ABI system can be used 
LaZe ^ r aChSr - system training L 

15 ABI svst " T*" " ainin9 ^ " Ser -—i°n in the 
15 ABI system itself, or can be teacher versions of student 

materials designed to assist the teacher in his/her use and 

textbooks. . Further. there can be teacher 

for example, to assist in generating student reports and 

st ,H:r ent : in tMs case the — 

object "Ho: 10 "! 1 ma " rialS '" 0 = i »- data 
objects 236 for performing this instruction, client systems 
for teacher access have agents unique to individual ZHZ 
Alternately, the flexible server structure of the ' 
« system permits administrative and teaching staff to perform 
their specific tasks on any computer system with enough 

acles rnecT UrCeE " °"~ * S '« '"«-" 

access, necessary system components are downloaded from 

servers to these temporary client systems. Thereby these 

ZIZ* 1 "* limited t0 SeSSi ° nS °" ™ Sterns 
Further, materials authoring can be done on server systems 

o lent systems, or on separate systems not interconnected 

with a given ABI system. To make authored materials 

available, they are transferred to and indexed in the 

35 appropriate server system databases. 

for the Ce t n r iVe , in,plementaCi °*= °f the functions described 
for the student client systems and the server systems are 
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also within the scope of this invention. For example, it is 
known to those of skill in the art that by the use of various 
technologies, such as remote procedure calls or messaging, 
the functions pictured here as grouped together and on one 
5 system can be divided and distributed if needed. 

5.2. System Hardware and Operating Software 

The system elements illustrated in Figs. 2A and 2B can 
be constructed on standard hardware platforms meeting the 
10 requirements discussed in this section. 

5.2.1. The ABI System Network 

The utilization of a network is central to the ABI 
system. It is important that students or teachers be able to 

15 access the ABI system from any adequate client system 

anywhere at the school, at home, and when away from home, as 
in distance learning. Networks permit students and teachers 
to participate at more than one school, and, further, permit 
delivery of homework and instruction to remote locations. In 

20 each case the client system must access student data objects, 
instructional materials, and ABI software from the network. 

Access to all system components is typically provided 
from ABI system servers attached to the network. It is 
preferable, to use a single large network server in place of 

25 several smaller network servers. In all cases, it is 

preferable to store the updatable student model objects on 
server systems, in order that they can be downloaded to 
whatever client system a student accesses. It is also 
preferable to provide read-only software and read-only 

30 instructional materials from server systems. The advantages 
over local storage of these elements at client systems 
include simplification of version control and of access 
control. The student is freed from insuring the client 
system accessed has all the necessary software and manually 

35 transferring if not. Also server storage provides greatly 
reduced client system storage requirements, in view of the 
large amount of software and instructional materials in ABI 
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systems. However, in systems with limited communication 
bandwidth, it can be preferable to cache recently used 
software and instructional materials in order to shorten 
response times. 
5 Networks suitable for an ABI system can be of any 

configuration and protocol that meets the system's client- 
server communication and object transfer requirements, where 
the client is either a PC or a network computer ("NC") . 
Suitable networks can be private or public. The preferred 
10 ABI network in the case of PC clients, according to current 
technologies, is an intranet running the TCP/IP protocol 
suite and utilizing ATM technologies as appropriate. in one 
implementation, utilizing a single large server, such a 
server can be directly connected to the intranet or Internet. 
15 In another implementation, utilizing several smaller servers, 
these servers can be connected into a cluster by a LAN, 
preferably an Ethernet LAN. m this case, the ABI network is 
built from a local Ethernet LAN with remote connections to 
telephone lines using 28.8 Kbps modems, or other network 
20 links. In an alternate implementation, the network can by 
visible to the public Internet if adequate security systems 
(firewalls) prevent unauthorized access. This can make wider 
access available at lower cost than by switched telephone 
remote connections. 
25 The ABI LAN can be further connected to other ABI LAN's 

and to other networks, such as the Internet, by standard use 
of routers and gateways. An exemplary protocol for an ABI 
System network is TCP/IP supplemented with a client-server 
protocol such as HTTP and object transfer protocol, such as a 
30 multi-format mail protocol. ATM technologies are used as 
appropriate. Remote clients are expected to be indirectly 
connected to the ABI network. These connections can consist 
of routing over a public network cr direct dial -in 
connections. These connections can be of either low-speed or 
3 5 high speed. The ABI system can be implemented on networks 
such as ©Home (Mountain View, CA) , which use a combination of 
ATM, cable modems, TCP/IP and other technologies. The ABI 
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system can also be hierarchically configured on new network 
topologies for distance learning in areas with limited 
communications infrastructure. Primary central servers with 
ABI software and instructional materials communicate with 
5 remote secondary servers over broadband satellite 

communication systems. Student clients connect to the local 
secondary servers through wired or wireless means. 

5.2,2. The Client System 

10 Considerations governing the choice of client hardware 

and operation software are described next and are followed by 
a preferred client node system according to today's 
technology. These consideration depend on the content of an 
ABI system and the facilities of technology. As the content 

15 of a system changes and as technology evolves these 

consideration dictate that the preferred system will change. 

The client hardware consists of client input/output 
("I/O"), client CPU and memory, and client network access. 
Turning to the client I/O requirement, standard input devices 

20 such as keyboard and mouse, or other pointing device, are 
preferable. Color graphics output capability adequate to 
support partial screen animations is preferred. Sound 
generation and output are preferable on ABI client systems. 
Text -to- speech conversion can be done either in software or 

25 in hardware. When economically available, full video 
capability, for example by providing video decompression 
hardware such as MPEG decoders, and speech recognition, for 
example with hardware assists, are also preferable. 

The ABI system is also adaptable to special I/O devices 

30 appropriate to special student groups, such as the very young 
or the handicapped. These include, for example, simplified 
keyboards, touch panels, VR devices, and so forth. 

Client memory must be sufficient to contain resident 
operating system components, resident ABI executive software, 

35 and dynamically loaded segments of the student data object, 
instructional materials, and code. High performance CPU's 
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together with high performance graphics hardware and memory 
is preferable to enable more advanced presentation effects. 

In embodiments where part or all of the ABI software is 
implemented in special languages, hardware or software 
5 assists for these language are preferable. For example, 
where such a special language is JAVA"* (Sun Microsystems, 
Mountain View, CA) , JAVA™ chips, which enhance performance of 
the JAVA™ interpreter, are preferable. The ABI client node 
in certain embodiments can be a JAVA™ enhanced network 

10 appliance adapted to Internet communication access and the 
HTTP V2.0, or equivalent client -server protocol. 

It is preferable to eliminate the need for permanent 
data storage devices, such as hard disc drives, by 
downloading all client software for each session. This 

15 allows the use of low cost 'network appliances' as student 
client computers. However, for those clients with lower 
speed network access asynchronous downloading can be used. 
Storage devices such as hard drives can be preferable for 
prefetching or caching of read-only software components in 

20 order to reduce start up time. To download 4 megabytes of 
software using, for example, a 28.8 Kbps modem takes over 20 
minutes. In the case of caching on student client, standard 
version control methods known in the art are necessary to 
ensure that only up-to-date software and materials data are 

25 used. If an element is found to be out-of-date by querying a 
server, the current version is downloaded. The student data 
object is not be cached between sessions since it can be 
accessed from other client systems. 

Client communication hardware can be adapted for either 

30 local or remote attachment. Local access requires network 
access hardware; remote access requires a communication 
capability. This invention is adaptable to lower speed 
access over switched telephone line services, preferably 
using 28.8 Kbps modems or ISDN interfaces (64 or 128 Kbps). 

35 These bandwidths are adequate for sessions with materials 
using only voice and limited animations. Prefetching and 
caching can be required to make fullest use of other 



WO 97/44767 



PCT/US97/08687 



materials at this bandwidth. This invention is also 
adaptable to high speed access over any available high speed 
links, such as Tl (1.5 Mbps) , T3, ADSL telephone lines, or 
cable modems (several Mbps) , or other means of high speed 
5 access. These bandwidths permit full access to materials 
without limitation. If economically available, high speed * 
access is preferred. With greater communications bandwidth, 
the on-screen agent can appear more life-like. 

Standard client software includes an operating system 

10 and communication software. The operating system preferably 
has interfaces to client I/O devices including communications 
capability and network access, such as a TCP/IP stack. ATM 
interfaces are present if necessary. Preferably, it also has 
means for establishing sessions with servers, for providing 

15 file server services, and means for security as specified 
shortly . 

The implementation language or languages of this 
invention preferably have several features related to the 
implementability , maintainability, and extensibility of ABI . 
2 0 The implementation language preferably provides a degree of 
modularity similar to that provided by object-oriented 
programming languages. It preferably provides means of 
dynamically loading across a network and executing additional 
software segments during program execution. It preferably 

2 5 provides means of accessing all input devices and of 

controlling the output devices in a high level display object 
fashion. It preferably provides a threaded or 
multiprocessing capability. Less preferably, this invention 
can be implemented in any computer language, including 

3 0 assembly language. 

In view of these needs and in view of the current 
technology, a currently preferred client system is a IBM type 
PC with a Pentium™ 120 Mhz processor, 16 MB memory, a 1 GB 
disk drive, a Soundblaster compatible sound card with 
35 speakers, a medium performance graphics card such as a 
Diamond Stealth card with 2 MB of graphics memory, an 
Ethernet card or a communication card and a 28.8 Kbps modem, 
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and standard keyboard and pointing device such as mouse The 
operating system is Windows'- 95 with network services 
provided by a World wide Web browser equivalent to Netscape 
2.0 or better and capable of running Java™ applets. Java" 
5 together with standard system and graphics classes is the 
implementation language . 

5 -2.3. The Server Sysfceig 

The primary function of the server systems of this 
10 xnveatioa is to store databases of executable software 
elements, of student data objects, and of instructional 
materials. The latter two consist of heterogeneous and 
structured elements. These elements can be stored in a 
relational database such as supplied by the Oracle Corp or 
15 the Sybase Corp.; they can be stored as specialized data 
files; or they can be stored in an object-oriented database 
system such as ObjectStore (Object Design Inc., Burlington, 
MA) . The operating system of the server nodes must support 
whatever database systems are selected as well as network and 
20 application server software to access the databases 
Application database server software of this invention 
preferably provides database access and version control and 
downloads database elements on client request. 

The preferred server hardware and software can vary 
25 widely depending on the number of clients to be 

simultaneously served. This number can vary from 20 at one 
school to more than 5000 across an entire school system The 
number of servers and database distribution across a server 
cluster can be adjusted by means known in the art to satisfy 
30 projected peak loads. A suitable medium performance server 
system can be configured on a high end INTEL Pentium or DEC 
Alpha system with adequate memory and disk space. Windows™ 
NT xs an adequate server operating system, and Internet 
server software similar to that from Netscape is adequate for 
35 network access. The preferred database is an object oriented 
database such as ObjectStore. in this embodiment 
application database access uses a common gateway ' interface 
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("CGI") program also providing database access and version 
control. The CGI access program can be implemented in C++, a 
suitable object oriented programming language capable of 
accessing interfaces to ObjectStore databases. 

5 

5.2.4. ABI System Security 

Security and access control present additional client 
and server requirements which are importantly part of an 
implementation of this invention. Security and access 

10 control can be maintained by careful selection of management 
policies, security software, and security hardware. These 
elements are described in this section in the order of 
authorizing and controlling access, operating system and 
network security requirements, and implementation language 

15 issues . 

In a preferred embodiment, the primary means for 
authorizing and controlling access are passwords. System 
management of passwords preferably includes ensuring that 
user passwords are secure, not easily guessed, and are 

20 periodically changed. This invention is also adaptable to 
any other means of access control, including for example, 
passive and active identification cards and recognition of 
certain personal characteristics, such as voice. Access 
protection can be preferably provided by limiting access to 

25 system resources - database and file - based on a user's 

password. For example, access protection can be implemented 
in the CGI application access programs. 

The operating system in clients and servers of this 
invention is preferably of a tested security level. This 

3 0 base security can be enhanced by a variety of techniques and 
tools that can provide increased levels of security for 
additional investments. Such techniques and tools include 
firewall machines, that is dedicated network gateways that 
filter incoming and outgoing packets according to content 

35 criteria, and monitoring software, such as tripwires, that 
observe system events for suspicious combinations. Further, 
encryption can help protect sensitive and valuable data from 
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illegitimate access by those without the kev p 

known x„ the art. such as the Rivest-Sha„,ir- M el ra an <KSA> 

. Ca\ c 3 ::L:r ithm or the - — — «- £~ 

For those embodiments and material e = - ... 
ex~cutahl* materials data that download 

ex.cutable code segments, the implementation language 
importantly should address t-h* ■ «=»nguage 

created L security exposures thereby 

10 can eXamPle ' 3 maievole nt and knowledgeable user 

r eg c i:L e t e or modify the downioaded — - 

illegitimate operations within the client svst-o™ 
restricted infection from the serve I ^J^""" 
is preferable in these embodiment because it Z ^ 
significantly addresses these problems and further 

izizzzir constantiy being made - E ^ 1SS <* 

security measures include limiting access to client system 
resources, particularly the file system and network 
connections, preventing downloaded software f rom . spoofing , 

20 0 "e / 3re ' ^ Pr ° Vidin9 byt — e verification t ^ 
20 c 0de for possible security violations. Any implementation 

language for an ABI syste m preferably offers sLlar or 

improved security features. 
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5.2.5. ABI Sv3tem Usage Metering 

Each student receiving ABI instruction on a specific 
topic is presented with a combination of instruction, 
utilities, and on-screen agent interaction. Because the ABI 
5 System provides individualized interactions, each such 

presentation comprises a unique mixture of visual, audio, and 
software elements. Also, each user's consumption of hardware 
and communications resources differs. The ABI System 
therefore preferably incorporates metering technology to 

10 track the utilization of the different elements of the ABI 
System. Such information can be used to determine payments 
tc the creators or providers of these elements and/or as a 
basis of charges to the user. Further, this information can 
be used by ABI System providers to determine absolute and 

15 relative usage of various components of the system. Parties 
interested in such information include providers of server 
hardware, application software, networks, instructional and 
tool content, authoring tools and on-screen agent animations, 
scripts and utterances. Finally, this information can be 

20 used to monitor system performance and utilization. 

Current metering technology does not provide the 
detailed measure of usage that is advantageous to ABI System 
providers. Such technology adapted for network distribution 
of content is available from, e.g., the IBM Corporation, as 

25 the IBM Cryptolope, and Electronic Publishing Resources 
(EPR) , as the Digibox. 

The ABI system comprises hardware elements, standard 
software elements, special application software elements, and 
content elements typically supplied by different providers. 

30 Hardware elements include networks and other communications 
links, server CPUs, and disks. Standard software elements 
include operating systems, database packages, and 
communications software. Application software includes 
instructional playback engines, schedulers, and agent 

35 software. Content includes instructional materials, 

utilities such as dictionaries and encyclopedias, and on- 
screen agent animation and utterances. 
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AM metering comprises the elements of the metering 

and of the preferred methods of monitoring. The pricing and 
compensation model is preferably provided by system 

^M 1S " at0rS - *" Meterin3 S °"ware ,-The 

MOS-) measures usage and maintains usage data in a 

L! " rU "" Ci,ne ° n °» and 

the system server, s). m the student client, the ms is a 

10 component function of the Session M anager with inputs dived 

from the Executive Software. Agent Software and the Materials 

System;" 6 ^ ^ ^ ' - 

cne system Manager Software. 

To facilitate metering, each ABI element is tagged with 
15 an ownership identification code similar to a producf 

barcode The first digits indicate owners subseguent 

To VT^ SP£CifiC COmPOnent " AS ele^ntHo" 

through the system from server to client and within the 

client, the use of these elements is metered. m the client 

Jo mir MatSrialS Engine USeS S °-r S hi P tag 

to meter content. The agent utterance generation and v lsual 
display generation uses tags of the data snips to track 
ownership of on-screen agent actions. The session and screen 
manager keeps track of which element of the screen is £™ 

usao. lnV6nti0n " adapt3ble t0 a ^ -thod for measuring' 

usage appropr.ate to the particular component elements. m 

tL H f ° Ur Pref6rred meth ° dS ° f U "9e are by- 

time by-.nteraction. by - unit and by _ packets . The Y 

method xs performed at each server to track usage of 

30 communications resources. This method is preferably 

implemented by using a count of the number of bytes 

transferred in order to estimate the number of packets 

transferred. The ABI application software keeps track of 

counting the number of bytes transmitted to the user 

35 according to content type (i.e. system packets, instructional 

materials, or elf animations). Then based on the network 

transfer protocol used, the byte count may be converted to a 
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roughly equivalent packet count. Alternatively, the byte 
count may be used directly as a measure. In the case of 
multiple servers, each server sends a summary of packet 
transmission information to a unique metering utility server. 
5 The by-elapsed- time method is performed at the client. 

In this method, the Screen Manager at the student client 
keeps track of the multiple simultaneous on-screen activities 
such as instructional materials, the elf, and the dictionary. 
This method preferably further monitors the active element of 

10 presentation, that is, the element performing an action or 
the focus for student input. In a more limited alternate 
implementation, the connect time of the student may simply be 
measured at the server. To avoid network problems such as 
lost connections and missed session signoffs, the System 

15 Manager application at the server additionally periodically 
obtains session statistics during a user session, either by 
polling the student client or from timed client downloads. 

The by-units method is preferably implemented either at 
the server or at the client. This method measures the 

20 content accessed by the client. For example in the case of 
instructional materials, the server records the specific 
lesson accessed while the client records the completion of 
exercises. In the case of on-screen agent actions, this 
method records each completed display or animation sequence 

2 5 or utterance. 

The by- interaction method is preferably performed at the 
client and records both fine-grained interactions, such as 
clicks and keystrokes, and the medium grained interactions, 
such as completed responses and metarequests . This method 

3 0 when combined with the by-time or by-units methods provides 

an indication of the relative level of interactivity of the 
instruction. For example, during use of instructional 
materials, the by-interaction methods measures the actual 
usage of each resource such as agent processing. 
35 Usage of the different ABI elements is preferably 

metered by an appropriate method. The methods described 
abcve are not uniformly advantageous to different types of 
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presentation, a graphics designer might seek compensation 
based on the amount of time a design is on the screen, while 
the instructional designer might seek compensation based on 
the fraction of the lesson completed. Accordingly, the by- 
5 e.apsed-time method is preferred for graphics designers, 
while the by-units methods is preferably used to record the 
publisher and lesson for instructional materials, the type 
and author for tools material, and the creators of the 
animation and utterance for on-screen agent actions. The by- 
10 packet method is preferably used to record the source of thl 

or Znl";"^ SUCh " aCCSSS tD StUd6nt °^ect database 
or downloading of application software, instructional 

materials, or animation data snips. The preferred metering 
methods provide infection needed for the t yp ical owner 
15 compensation and user pricing models. 

In its preferred embodiment, system administrators 
provide pricing and compensation models. This invention is 
adaptable to many alternative such models. One user pricing 

20 11 " •? baS6d ° n ^ ° f ucticn being 

20 made available, similar to pricing in the cable industry 
Another is based on connect time, " as frequently done for' 
internet access, or on type and quantity of information 
retrieved, as is common for on-line database access 

25 ^ illUStrates an ^emplary model for compensating 

25 providers of ABI system elements. Compensation is based on 
metering data and an allocation model . The f our meter i nq 
methods are applied selectively to different providers The 
allocation amounts 'a', <b<, <c< and <d' are externally 
determined . 

30 



35 
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TABLE 1A: TYPICAL MODEL OR USING METERING INFORMATION 





PROVIDERS 


COMPENSATION FOR PROVIDER 




Billing/Network 


'a' percent 




Server Hardware Providers 


'bl' percent divided up according 
to number of packets with up to 

pcicexiL- aaaiLlOIlal IE tOCdl 

number of packets exceed a 
predetermined level 


1 ft 


Instructional Materials 
Providers 


'C percent divided up according 
to units 




Calendar/Schedular Provider 


Up to 'd' percent based on level 
of interaction 




Animation Providers 


'e' percent divided up according 
to 'time on screen' 


15 


ABI Client/server Software 
Provider 


Balance of revenue 









5 -3 . The Inatructional Interface 

20 The ABI system has interfaces for students, teachers, 

and administrative staff. Materials and software developers 
can have specif ic ABI system interfaces. Alternatively, such 
development can occur on separate systems followed by 
indexing and loading of the developed components into the 

25 appropriate databases. 

5.3.1. Student -System Interface 

This section describes the structure and design of the 
student -system interface. This description is directed to 
30 the currently preferred implementation of this interface by a 
series of visual display screens. However, this invention is 
not so limited, and adequate alternative technologies, for 
example voice output with speech-recognition input, can be 
used to implement this design. 
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5 - 311 - E*ctp1«tv stnd.„t <=---,-„ 
During materials presentation, a student sees screens of 
one or .ore screen types, aU formatted in accordance " 
the previously described desi 9 n principles. Screens are 
5 structured as a hierarchy of areas and subareas with text 

the sT " imati0n ' ^ ° b ^«= ^Pl^d'in 

the subareas. objects can either be for display only „ 

permit student input or interaction. 

An important screen type is a task screen. Task screens 

10 are used for materials presentation tasks such as hdlk 
assignments including problems, programmed steps, maste! 
quizes, and drills. Tab le x and pig 3 ^ «^ 

opponents of a task screen appropriate for elementary ^ 
education. Reference numbers in Table ! are from v£ 3 m 

« 9-eral, i„ system area 30 2 , the session manager presents 

a°cc::r M 7 uali2ea as icons psnu ^ «» ~ « « ^ 

access AB! system facilities, m materials area 304 object 
presentation including description, placement and moment 

20 Indt"" 6 ' ^ mat6rialS deSi3 "« in "» P».««" -1 

20 and sequencing sections of the materials data. The materials 
eng lne interprets these specification at run time to send 
display objects for display. Ia agent area 303, agent 
behavior processing formats predefined parametrizable 
objects, including resolution of object parameters 

» representing selected agent personae . The output from all 
selected and formatted dispiay objects is sent to the 

rrrstiir uare ana oper - in3 — <« — ^ 
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TABLE 1: STUDENT SCREEN DISPLAY ELEMENTS 



REF 

NO. 

301 
5 302 
305 



306 

309 
307 
308 

310 



10 



304 
15 311 
312 
313 
314 
315 



20 



25 



30 



316 



321 
317 

309 



303 



318 
319 



320 



35 



NAME OF ELEMENT 

Screen 

-System Area 

-Customizable 

--File System 



•-Scheduling Toolbar 
j- -Monthly Calendar 
! --Clock 

-Toolbar 



Materials Area 
Open Book 
--Section Tabs 
--Exercise Area 
-Header 
-Presentation 



- - Input 



--Navigation Buttons 
--Student Action 

-Sched . /calendar 



On-screen Agent Area 
j - -Audio 

-Text Message 
-Visual Persona (e) 



--Agent Request 



DISPLAY OUTPUT OBJECTS 

Entire ABI display area 

Graphics and text 

(can be animated) 
Book icons with titles 



Graphics 
Dynamic graphics 

Icons of tools (for 
calculator, word 
processor, dictionary, 
communications, etc . ) 



Graphics of book outline 
Graphics with text 
Text, graphics 
Text, graphics 
Text, graphics, 

animation, video 
Interactive objects 

(buttons, etc . ) , 

filling in text, 

dragging, drawing 

Graphics 

Icons for Submit, Help 

Graphics for calendar 
with text of student 
schedule, icons for 
scheduling options 

Synthesis of spoken 
utterance 

Text utterance 

Appearance integrated 
with audio, graphics 
background effects 

Interaction 

with other screen areas 

Agent meta- request icon, 
generates pull -down 
menu of meta-requests 



In more detail, the task screen of Fig. 3 includes in 
system area 302 student customization area 305. In the 
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T h TZHT i0n area ' StUd6nt diSplay *iven by 

the agent as rewards for student accomplishment. Tnese 

atilT ±nClUde t6Xt ' " illUSt ~ te * *»«. graphics, or 

animations. Another part of the system area is file system 

o 2TJ 06 diSPla/ing aCC6SSible fil6S " in a "book- 

c-shelf" metaphor. This invention is adaptable to other 

Z7o2 SyStSm diSPlay meta P h — "ere, for example, 

ile ZJ C ° nS rePreS6nt 9 file PerS ° nal t0 the a 
file holding ongoing instructional materials, a file of e- 

10 mail, and files for tools such as a dictionary and group 
■ctxvxty. Below file system toolbar 306 is tQolb J ^ for 

Z l TlTT nt has access to - Illu ™ - *■«». 

for a calculator, a word processor, communications, and 

starfish, a general purpose language tool 
15 "Starfish" are a visualization tool for semantic 

networks that can be available in an ABI System. A semantic 
network typically consists of nodes linked to other ™£ 
semantxcally significant and named links. A starfish tool 
visualizes such a network for a student by displaying the 

links displayed as arms of the starfish. For example, a 
dictionary represented as a semantic network might include a 
node for each word with links to similar words, opposite 
words, root words, and so forth, a dictionary starfish 
25 displays a word in its body. Each selectable arm is labelled 
with a category - synonym, antonym, pronunciation and so 

toT'.- 01 ^^ 9 ° n " ^ CaUS6S desponding word(s, 

to be displayed or spoken. Such a tool provides integrated 
display of diverse knowledge structures for a student 

The scheduling/calendar tool is an important tool that 

is always available. Exemplary icon 309 illustrated for this 
tool h lendar part 3Q7 ^ ciQck Seiect _ on 

each of these parts brings up daily and monthly scheduling 
functions. These function can either prescribe the student's 
35 next activity or pennit choice where the student has excess 
time or demonstrated personal scheduling ability. 
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In materials area 304, instructional materials, tools, 
and communications materials display their content. 
Illustrated in Fig. 3 is page 3 of an exemplary mathematics 
homework. Instructional materials are advantageously 
5 structured as a book of exercises and items, emulating 
current textbook and workbook practice. In this case, 
section tabs 312 permit the student to navigate the homework 
book by sections, and page buttons 321 permit the student to 
navigate the homework book by page. Also present is toolbar 

10 317 of available student actions, which for homework includes 
submit and can, if the agent permits, include help and hint 
requests. The format of a materials page is advantageously 
standardized. An exemplary standardization has header 
information 314, presentation 315, and interactive input area 

15 316. 

On-screen Agent area 3 03 allows the student entry of 
meta- requests and allows the agent to display synchronous or 
asynchronous meta- responses . This exemplary area illustrated 
in Fig. 3 comprises meta-request button 320 which the student 

20 can activate to display a list of currently available meta- 
requests. Student metarequests are generally allowed by the 
ABI system at any time. Requests may occur at the beginning, 
middle or end of instructional sequences, in choice screens, 
in tools, in group communications, or in the calendar. Agent 

25 metaresponses are individualized and context dependent. 
Typical metarequests are illustrated in Table IB. As an 
example of context dependence, if the request 'Where am I?' 
occurs within a lesson, a graphic representation of* the 
lesson will be shown,- whereas if the same request is made 

30 between lessons, a graphical representation of previous 
lessons and new lessons will be shown. The agent software 
attempts to respond based on the user's intention and uses 
the context and previous requests to infer the reason for the 
current request . 
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TABLE IB; TYPICAL METAREQUESTS 



10 



15 




Temporal: Time remaining in 
current unit /session/days work 



Marker: name and description of 
current lesson/exercise; relative 
position in lesson 



Pause: Repeat of instructions- 
summary of what has been done 



Performance: summary of work in 
current lesson 

Choice; Branch to take. e.g. 
which lesson to do next 



'How far along?' 'How much 
longer? ' 



'Where am I?' 



'Say it again. ' 
another way. ' 



'Say that 



Purpose : 
lesson 



function of current 



Resources: assistance available 
at this point 



'How am I doing?' 
What do I do next?' 
'What is this for?' 



' I need to hint , 
help. ' 



' I need 



The instructional materials notations provide a 
framework for the agent software to keep track of the 
20 relative position of the student in the lesson. This same 
information is also available to the teacher for lesson 
preview and annotation. 

The remainder of area 303 is for agent meta- responses 
wh 1C h importantly have multi-media structured into personae 
Illustrated are text message 318 and visual persona 319 that 
typically includes animation. Also possible is audio output 
either text- to- speech or generated from audio files. 

Fig. 4 further illustrates an exemplarv screen 
interaction between the materials and the agent Fig 4 
30 shows only the content of materials area .501 and on-screen 
agent area 502 of the complete display screen of Fig 3 A 
mathematics homework material is displaying item presentation 
503 with input selection buttons. The student has selected 
wrong input button 504. At this educationally significant 
35 event, the materials send to the agent several messages 
generated by notations in the materials data. m response 
the student's agent has chosen to act as illustrated. First, 
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it displays text 506 of the rule violated by the student 
answer. This text was sent to the agent by the materials in 
an event message for its use. Second, the on-screen agent 
points 505 to the screen location of the error. This 
5 location was also sent to the agent by the materials. Third, 
perhaps in response to a previous high or increasing error 
rate of the student, the on-screen agent presents a meta- 
response 508 commenting on the pedagogic nature of the 
student's error. Further, it activates a persona 507 to 

10 engage the student's attention. This persona can 

advantageously include animation, audio, and speech output of 
the displayed text. Thus, the agent software integrates 
speech utterances, visualization, display of text and 
graphics, and animation into a persona display for 

15 highlighting an educational event that the agent determined 
important based its processing of the current input, past 
student inputs in this lesson, and the student's pedagogic 
model generated over several sessions. 

Other screen types are of importance in an ABI system. 

2 0 For example, tools can have distinctive screen types 

appropriate to their nature, such as a calculator image. 
Choice screens can be used at the transitions between 
instructional sequences. They summarize what instructional 
materials have just been completed, announce status 

25 information, and list any materials choices now available to 
the student . These choices can related to instructional 
materials to be undertaken next or to optional, non- 
instructional materials, such as exchanging credits for time 
in a game room, access to e-mail, and so forth, that can be 

30 available to the student. 

5.3.1.2. Student Actions 

The inputs of a student or other user of the ABI System 
are preferably classified as requests, meta-requests , or 
35 data. A student request is an input directed to materials or 
to the system seeking a specific action. Student data is an 
input responding to a system request for information. For 
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example, student requests include an input to th. . . 
-art the calculator is . request, or an n ut t Z «1" 
-t.r»l. to submit toileted homework to the ^ 
However, input of numbers into the cal m , » Ceacner - 
. invention is adaptable to a further L^of J* 

-put. termed semiotic. in which the^tj en e e" s e. In 
"9ns or symbols to provide input. Semiotic inpu is 
particularly advantageous when the a =.n, 

to report his f™,<„! 3 "»!<"">" the student 

10 educational ven: M 9 e:i J""™ 1 *"- '« • Particular 

a 3 ent seeing ^cif^ 'iTY" " ^ 

to the agent seLi g L l .ng l"^ 1 " ^ 
meta-request. 9 CUr " nt aerials is a 

Displays produced by this inv.„n 
» classified as applications response "^"^ 
*««*». Application display occurs up r re T SeS ' " 

20 initiative. For example rh!' ° CCUr ° n SyStem 

- . tod avaiiabiTt 1 :;:^::::-— — ; ::;lt ™ « 

-splay the adequacy or J^^Z nT £T 

-quest, or asynchronously, generated b v n t HTZ T' 
agent £ rom the materials. O uestio„ s are a pa i " a r ° L , 
response or meta-response which seek ta „ h . ° £ 
30 Questions engage the student L form 1 1, ^ ^ 
- integra! component of the ^Z^T " 

interact^ o"^™^ ZT^ «" °* 

auring a mathematics homely £ °= C " r 
3S responses and requests T" 

and meta-requests and alS o how thf IZ ^ZT^ ' 
observ.t.on of the student's current situation and its 
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contact with past student history, is able to guide the 
student better than the materials alone, which are only aware 
of the current context. This display is individualized to 
the student's current and past performance and preferably 
5 uses realistic, life-like on-screen personae to engage the 
student. Displays from the ABI System directed to the 
student are indicated generally on the right; inputs from the 
student directed to the ABI system are indicated generally on 
the left; and time increases downward. The diagonal arrows, 

10 as at 401, indicate that the message at the arrow's tail 
leads to the message at the arrow's head. 

This illustration begins with response 402 from the 
mathematics homework materials suggesting that the student 
use the calculator tool. The student responds with request 

15 403 which selects the calculator icon to request the 

calculator. Calculator application 404 is then activated and 
displayed by system components. Data 405 is a student input 
to which the calculator tool, and response 406 indicates a 
student error. This response was apparently not sufficiently 

20 informative, and data 407 is a repeated incorrect input to 
the calculator topi. The student's agent observes the course 
of this interaction as the calculator tool sends event 
messages to the agent . Upon observing the repeated wrong 
input, the agent intervenes with a asynchronous meta-response 

25 408 summarizing recent student inputs. The student responds 
with meta-request 409 to the on-screen agent for help, since 
the tool responses were apparently not sufficiently helpful. 
The agent's synchronous meta-response 410 suggests for 
consideration an entirely different approach to correct the 

30 student's recent error. 

5,3.1.3. System Responses 

The following general principles preferably guide system 
and materials responses and agent meta-responses . First, 
35 some display is to be made on every user input, if only to 
echo a character or mouse click. Second, a user is never to 
be left in doubt about the status of current tasks. For 
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r amPle ' °" SyStem P»'"»"y Wrt*. task specific hints 
or suggestions if no user j . nts 

adaptively determined ™ L a in 8 P " iod 

mira, all responses reflect the 
current context. 8CC tne 

5 be ajptet:; 1:1°™^- aspects ° f system reSp ° nses — 1- 

be adapted to the particular student audience, from 
elementary to adult education and including special classes 
of students This tailoring can be set hy \ h l t I h ^ 

io e : ™; e ve ia staff • one adaptabie — is ^ 

level and the language of system responses - for examole th. 
vocabulary Qf ^ services( ™. - 

aL'ptabi! I Pref6rab1 ^ ^^ustable. Another important 
adaptable aspect xs the type of pe rsonae of the on _ screen 
agent is preferably adjustable. The types of •„ 
15 level of iokes an H * u encouragement, 

student n ' ClOBel y on the -tended 

student populat.cn and are advantageously adjustable. 
Further, tool8 are alsQ adaptable Each ^ 

2S particui a<itiiti0n ' Cer " in ^ Smem have 

part.cular preferabie and advantageous features. These are 
listed subsequently. Importantlv m.,. 

preferable .h - "POrtantly, meta-response selection is 

preferably shaped ln vl ew of the studenfs past baselines of 
speed performance accuracy, modality, as specific to the 
type of materials. „ovi„g average functions, in which recent 
30 values are gi ven higher weight than earlier values can be 
used advantageously to genera baselines of performance L 
The tiding of meta-responses is preferabLTsl "„ 
student data ob 3 ect fields that record J amoun f h p °" 
asked for ln the past and past perforce provided 

Further, the pacing o, meta-responses is advantageously 
context sensitive. P or example, remediation should b! ' 
offered only ln case of repeated error or hints offered 
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asynchronously only after a relatively long period without an 
answer. Also, previous student interactions should be 
utilized. Repeated requests for hints should be noted and 
dealt with perhaps not with another hint but with 
5 remediation. The rate of prompts, advice, and hints should 
be adaptively adjusted on the basis of ongoing performance - 
records . 

The actual content of a meta-response can be adjusted to 
the current situation by filling in parameters from event 

10 messages sent from the materials. See infra. The 

Instructional Materials Interface Standard ("IMIS") is a key 
element in providing information to the student in response 
to inquiries such as "Where am I?». The system then uses the 
information in the IMIS to respond, "You are halfway through 

15 the drill and practice on fractions," for example. A student 
request for further information can elicit a graphical 
response. The graphical responses reinforce the virtual 
tutoring aspect of the system by providing a greater level of 
detail in a format similar to that presented by s human 

20 tutor. The format of the graphical method used is 

individualized to the student's past use of system components 
such as previous graphical metaresponses and tools such as 
the scheduler/calendar. 

The structural information for the agent software to 

25 construct graphical responses is derived from lesson 
notations, previous student performance, previous agent 
responses (including graphical responses) and the 
scheduler/calendar. Marker requests use general notations 
for the type of lessons and specific notations for 

30 prerequisites. Temporal requests use notations of expected 
time together with profile information on individual 
performance on that type of material. 

Typical predefined graphical response templates include 
roadmaps (perhaps in the form of a train and subway map) , 
35 charts, table, graphics and icon flowcharts. Colors can be 
used to the show relative levels of perf ormance, urgency or 
dependency of elements. In response to a 'How am I doing' 
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«que„t. green ^ indicate ^ 

simlerly. the shape of the icon „i 9ht indicate Us 
«eh as a diamond for caution or . 

5 Figure 12 illustrates an exemplary dialogue starting 

with the student metarequest -where a7l- ^e IT 

HZIT ~" a • ^™ 

h- U sonTIT' ™ " Y ° U « ~ through 
- Aess °n on adding fractions " Next- t ho o- 

10 request further inf oration. Preferably t„e 1 ^ 

"ill point to the nation on tto ll^TLT 

Oere.- „ the student then clicKs on L , °" *™ 

char . hha clicks on an element of the 

exa^e 1 39 ! nt fUrth6r inf °^tion. For 

example, ci lcklng on the Practice ^ 

15 me are S ponse containing a nowchart of 
activity, as generally illustrated. 

The calendar scheduler is^a^nTa^u^T^^r^ 
20 providing the following responses ThL 

schema esponses. These are controlled by 

schedule data contained in the student data object and are 
processed by the calendar/schedule tool 

1. Schedule reminder responses, which remind the 
student of deadlines for material c , , 
25 external activities, su ht n IT'lT ^ ^ ^ 

a class outing. 9 parent 5 -PProval for 

an orl ' <«Wi°n responses, which suggest 

an order of assigned tasks based on student history and on 
the assigned priority and deadline. 

30 task !ill T ;T? Stimate «*P°»se B , which estimate how long a 

asK r ed -° n tim±n9 iafM »"« ™> - Part of 

lnst ™ ct — * materials and on past relative 
performance for this student. 

35 erh emb ° diment ' the J"* astern provides for of 

35 scheduling initiative to be divided between the student and 
the system. Settable task schedule parameters permit at one 
extreme, the student to have complete scheduling Zl'rol 
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able to initiate and to exit any activity at will, and limit, 
at the other extreme, the student to work on only those 
materials that the system schedules. These schedule 
parameters include those controlling the tools and options 
5 available to the student while performing a given task and 
those requiring the student to perform background reading or 
remediation. Important initiative parameters include the 
scheduling values of task priority value and deadline. If 
the priority and date are not both "off", those tasks with 

10 greater priority and earlier deadline are automatically 
scheduled for the student. If these values are "off 11 , the 
student has control of task scheduling. 

The system of this invention also includes a central 
list of timed activities, perhaps stored on the server 

15 systems, to be performed by the system. These can include 
regular times for the generation and printing of standard 
reports or for the broadcasting messages concerning group 
activities such as spelling bees. These timed activities can 
be performed by scheduling software which compiles the timed 

20 activity lists and initiates the listed activities at their 
listed times. Such scheduling software can be similar to the 
"at" utility in UNIX operating systems. 

5.3.2. Teacher /Administrator- System Interface 

25 The teacher or teacher's representative plays an 

important role in the ABI system. The teacher uses the 
system to perform such functions as entering initial profiles 
in student data objects, assigning students to subgroups, 
previewing, annotating and scheduling assignments, reviewing 

30 and commenting on completed homework assignments, and 
reviewing summary reports. 

The agent of the student is also an agent of the 
student's teacher in that the student's teacher controls key 
parameters in the student data object, which in turn controls 

35 agent actions. The teacher customizes the ABI system by 
setting student data object parameters, assigning and 
prioritizing assignments, and customizing materials. 
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Important teacher accivities are inciudea ^ 

control 0 ThS CeaCher inUiali2 « — ~rei«. continuing 
control over important data in the student data object L 

student's access to the materials Thi-, 4 , 
IS tasks, remove tasks or modl£y their pr^itLl n °"" SyStem 

stuJt, Th ;r:«: n r t c :: cus r* e materiau — ^ «■ 

■ routine customization includes 

mod l£yi sequencing of instructional lessons, elements and 

students. " rePOrtS ' Sendi "3 ■»•« « 

4 . The teacher' s class management is aided by a 
facility to send messages, reminders, hints etc L r „ 
2S using the ABI system e-mail facilities 

The system can advantageously assist the teacher in 
homework management. Once the student completes and suL> 
a homework assignment, a printed copy can be ml r T 

30 : d e v n i the scudent - ^ h ™ =» > 

graded by the ABI system, if answers were provided as part of 
homework material The r.»,.t,. ^ P 

student l, h , ad " c °™><=hts for the 

student, lf homework is viewed online by teacher 

The system can advantageously also provide the t..rh. 
with summary and detail reports for 
35 These reports can be mediately available online or printed 
for later review. As known in the art, reports can ZZ „ 
both current and cumulative dat, on instructional ™ " 
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and homework assignments. The reports can also flag patterns 
of deficiency in a student's homework and problems in a 
student's instructional progress. in a preferred embodiment, 
these reports are generated from the database of student data 
5 objects on the server systems. 

Additionally, the teacher can be a student. A teacher 
can benefit from training in the use of the ABI system in 
general, in the procedures to customize materials, and in the 
characteristics of the particular materials used in that 
10 teacher's class. This training can advantageously be 

packaged as instructional materials directed to the teacher 
which are otherwise similar to student instructional 
materials. In this case, the teacher, like the student, has 
accessible materials, a teacher data object recording the 
15 teacher's progress and pedagogical characteristics, and an 
agent using this data object to guide the teacher's training 
and guide the teacher in the use of the system. 

Other actors, such as school administrative staff, 
parents, and researchers, can play a role in the ABI system. 
20 Administrative staff can have privileged access to certain 
data items in the student and teacher data objects and other 
system data, which permits them to assign students to 
courses, to assign students to teachers, and to establish 
instructional performance standards and criteria which the 
25 students must meet to complete their materials. This staff 
can also receive online or paper reports on the progress of 
students in the schools, the effectiveness of teachers, and 
the usefulness of the particular materials assigned. 

30 Reports and queries 

Generation of reports from databases, either relational 
or object-oriented, is a standard programming task. The key 
elements of this task are the selection of data to appear in 
the report, the way in which selected data is to be presented 
35 whether in a summary form, a detail form, or both, the format 
in which the data is output, and the layout of data output 
elements on the screen or page. 
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record "* *<* dnis "* t °™ "<J»ire several types of 

report, Perlodic reports ^ ^ ^ regular 

intervals. These vary accordin 9 to population: by 
* class. by gra de, oy school. They also vary ^ 
5 The reports can present/summarize complete sets of data or 

fulL^r e r eedi09 Stated limitS - *"» ° f "Ports 

ana ~ reP ° rtin3 ^ reC ° rd kSepi "9 -^irements, 

and. some are sent to parents. The ABI system also provides 

teachers and administrators with standard querying and report 
10 generating capabilities. report 

If appropriate, parents can also be actors in an 
em*od lme nt of the ABI sysCem . A student , s parents 
given access to certain fields in their student , s ^ * 

15 hi, " " n reC6iVe " Pid infor.ation on their 

15 child's assignments and performance. This information an be 

student recedes instruction and homework 

Finally, educational researchers can receive certain 
access to ABI systems in order to research the effectiveness 

: ^ fSi::r nal theories and methods - p ° r ™- ^ - 

paradM ^ °' various educational 

parades m certain instructional contexts by receiving 
reports relating to students pursuing materials constructed 
according to the paradigms of interest 

25 

5.3.3. Instructional Designer -System Interface 
Pcsigner Int>rf r e Ov er vi w 6 

Materials, in particular instructional materials, are 
authored by instructional designers. Authoring of materials 

30 can be done on the system on which the materials are to be 
used, or alternatively, on a separate system. Generally, an 
instructional designer authors materials including, for 
example, computer assisted instruction as known in the art 
computer assisted exercises such a homework or simulation ' 

35 and computer managed student instructional tasks which can 
evolve work with several materials. For all materials the 
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student's agent must be informed of the sections completed 
and skills acquired in standard formats. 

Designer Interface Details 
5 The ABI system provides an environment in which the 

student's agent is available to control materials 
presentation and guide the student to improve educational 
outcomes. This environment includes facilities to present 
assignments, assess responses, probe for prerequisites, offer 

10 assistance with tools such as dictionaries, and score unit 
mastery quizzes. The system can advantageously include 
conventional forms of homework, such as worksheets, as well 
as new types of homework, such as group -based homework, and 
formats previously too time consuming for teachers, such as 

15 criterion based rather than fixed length or interactive 
rather than prescriptive. Advantageously, the system 
structure can accommodate existing forms of computer assisted 
instruction by embedding such existing instruction in 
materials of this invention which contain notations and 

20 generate agent event messages. 

All authoring information below can be provided by the 
instructional designer, if preferred. In this case, defaults 
allow the teacher to enter only overriding information where 
needed. Alternatively, the teacher can be allowed or 

25 required to make various degrees of customization. In this 
section, the term "exercise" means a single unit of an 
instructional task. 

The run-time conversion of materials with descriptions 
of text, graphics and animation advantageously is performed 

3 0 by methods commonly used in interactive multimedia Authoring 
Software. Existing authoring programs let the author specify 
the attributes and behavior of multiple independent display 
objects through such programming paradigms as visual 
programming or scripting languages. For example, to display 

35 a button or other selectable object, a user of authoring 
software specifies such characteristics as position, 
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attributes such as size, shape, color, and label and the 
f uneven invoked by the object selection 

oroduT^ Widely US6d ° f the mailable 

products are Macromedia's Authorware and Director 

^heT triX ' S MUltim6dia T ° 0lb00k Md Aimtech ' s IconAuthor 

and otr 9 ^ ^ b6en d ° CUmented by thSir 

and other writers (Sims, Authorvare ^ awaria ^ n 

et al. ( WaOTe dia Director Design Guide,- Hill c/sina 
Multimedia Toolbook 3.0). * 
10 The process of authoring an instructional task ±» 

advantageously described serially to includ he steps 
of, for example, defining the task Hcf • P 

student inputs , ^ - - 
au^nt.ng the ials „ Uh notatims - ana 

in the authoring step of defining the task the 

written and preferablv a1 c~ = , 

prererably also a spoken format with alternat wo 
wordings as necessary £or spec . ai ciassss al ernatrve 

30 des lgn er also provides materiais nts . » e 

the education paradigm chosen. lnstructional 
appropriate for interactive instruction with feedback 

In the authoring step of defining each exercise the 

- zeroise, ^^r^TT.^ LV 
acouirea in accordance with schooi svsten, standard so that 

pr vlTr T in9£UUy CraCk StUd6nt P " f ™ ana 

provide helps, hints, and remediation Next fhe „ • 

35 provides student instructions for the exe" se JllZZ in 
both spoken and written formats. Hext, the designer 
specrf.es the presentation of the exercise by creating 
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graphics and text for the exercise and by specifying their 
display attributes. Next, the designer defines allowable 
user inputs, the data type of input, and the display areas in 
which input is allowed. Finally, the instructional designer 
5 provides hints. Hints preferably include optional hints 
offered on student request or student inactivity and hints 
offered by the agent in case of totally inappropriate 
actions . 

In the authoring step of judging student inputs, the 
10 instructional designer advantageously provides information 
including the following. First, the designer defines 
standards for the completeness of requested inputs and 
actions, possibly none, in case of incomplete inputs. An 
exemplary such action is suggesting the completion of omitted 
15 inputs. Second, the designer selects how inputs are to be 
judged and the possible judgements, for example, correct, 
near miss, incorrect, and so forth. Finally, the designer 
selects error responses to individual input elements or an 
entire exercise. These responses can be uniform or can 
20 reflect the specific wrong answer. Error responses include 
retry policies and selection of remediation paths. 
Remediation can be item sequencing options given on the basis 
of retry outcome. 

It is preferable that response judging be performed 
25 according to certain standards established for a particular 
implementation of this invention. Thereby, the agent can 
monitor the student's pedagogic characteristics in a uniform 
fashion across all standardized materials. Accordingly, a 
first step in input judging can be, where appropriate, 
30 checking the data type and perhaps range of an input. For 
example, checking that a fraction is entered in response to a 
fraction addition problem. Appropriate feedback can be given 
if the wrong data type or range is input. Single student 
inputs can include entry of a single text or numeric element, 
35 movement of a single display object, selection of a single 
display object, or drawing- a line or curve perhaps encircling 
a display object. 
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exemplary embodiment, this can he . / 

r, at -^^ e P erf ormed by providina a 

10 selects fr„„ , • . f^ceaure is advantageously 

selected from a l imi ted number of standard judgements ? 
correctness and partial correctness Thi « gements of 
communicated to the aoenf ■ ThlS 3 ud ^ment is then 

Patterns can include one nr 
more parametrized text- r- e ,„ro=.- e or 

-» . ejects to be moved bv rhp C h„j nT ,^ 

input"! ° bje0,:S ' ™ Ze '"versal, and tool 

c^^™'"* " aUCh0ri "9 step of defining task 
completion, the instructional designer advantageously 
provides information including standard, f„ 
» -d tas, completion. The «Lrr^T™"' 
-imum completion -fore allying the 

with Idditio a Y Uth ° rin9 iS • u 9~»*"9 the materials 

wicn additional notations for th* 

30 concern tas k and exercise sublet s" l £T """"^ 
^inition of the ^ew^* 

the aerials th at gyrate e m"" 7^™ 
>5 that reference control parameters setTL ag "* 
control of materials sequencing. The notations so entered 
communicate various types of information to the agent 
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Exemplary types include characterization of assignment by 
type of task, subject, elements of subject covered by task; 
comparison between exercises in same assignment by similarity 
and difficulty; the subject matter prerequisites to determine 
5 what the student may or may not have mastered based on which 
exercises are answered correctly; system knowledge 
prerequisites; and scheduling information such as average and 
expected time to complete. 

10 5.4. ABI System Software Structure 

This section describes in a more detailed fashion an 
exemplary structure for the software components previously 
described in a more general fashion with reference to Fig. 2. 
Subsequent sections describe particular components in an even 

15 more detailed fashion. The structure described here is 

exemplary. This invention is adaptable to other structures 
with other allocation of the functions of this invention to 
different modules. Such alternative structures are easily 
designed by those of skill in the arts. 

20 This section first describes the principal client 

software and data components and the conceptual hierarchy 
which reflects their interaction. Next, the structure of the 
executive software is described. Finally, the flow and 
processing of events through the system are described. 

25 

5.4.1. Student Client Software Structure 

Fig. 6 illustrates the principal client software and 
data components and the conceptual hierarchy which reflects 
their interaction. The links illustrated represent principal 

30 procedure cal or class dependencies among the components. At 
the root of the hierarchy is operating system 601, which has 
been previously discussed in connection with the preferred 
client hardware. The operating system ("OS") includes the 
operating system proper together with implementation 

35 languages and their class and function libraries with 

associated controls. At the next level is executive software 
602 ("ES") . This software collects a number of components 
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which customize the operating system co the requiremen£s 

au o ^sT" ^ alS ° extMd U ' i£ p - 

fa Uitv o Cr6atl0n " Pr ° CeSSed thr ° U9h - K «"»! 
facility to insure that the student accesses only permitted 

5 materials. The ES software also provides a preferreT 

animation facility and controls client startup 

clienrT SCree " mana9er 603 is alm ^ *> r «*« °» a 

\:r :' ™" component partiti ° ns °» — i-o ^ 

areas for the principal system components, as has been 
10 generally illustrated by the exemplary screen o, Pig , and 
controls the system area, area 302 in pig. 3 Thus ' L 
student selection of an icon presented in the sy! m Zl 
revesting the ES to start the function represented, it "urn 

» zziT h ^-^^ «>■> «- S t»dent i 

w'th OS ta!r" tad " thiS fUnC " 0n bS£ °~ 

with OS task creation. The system manager also presents 

whatever reward graphics and animation the student has Ln 

granted access. These functions are performed by calling the 

object level l/o facilities of the OS and ES 

20 ob j ect 1 6 S i a T ai 7 h PreEent ~ 3 Cl±ent SYStSm U StUde " ^ 
object 6H for the student in session on the client This - • 

oblect, preferably containing all the permanent read-write 
data in the m system relating „ ^ down 
from he server student database when the student logs out! 

25 the client. Preferably, this object is divided into 
subtypes, and those only those subtypes referenced are 
downloaded as required. In an alternative implementation 
"here the permanent student data is stored in more than one 
object, each object is downloaded from its database as 

30 required. As indicated by double headed arrows 608 and 61, 
various mi system components reference and update data items 
in the student data object. 

schedule/calendar component 607 is important for 
management of the student by other system actors, such as 
3S teachers and administrators, it is a tool that is always 
active whether or not it has a screen presence. Activities 
for the student, represented by activity description 
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priority, and deadline, are entered into the 
schedule/calendar data subtype of the student data object by 
teachers or administrators. Optionally prerequisite 
information can also be entered into the schedule/calendar 
5 subtype. This component uses this schedule/calendar subtype 
data as well as the time expected for the student to complete 
an activity, as determined from the student's past 
performance also stored in the student data object, in order 
to determine whether certain activities, such as particular 
10 instructional materials, should be automatically started by 
the ES without any student intervention. In cases where the 
student is permitted greater initiative, the 
schedule/calendar can permit OS task creation requested by 
the student, as by selecting an icon. This component also 
15 provides, on request, prioritized lists of tasks for the 
agent to present to its student when the student asks for 
advice on what to do next. It uses priority and deadline 
information and time-f or-completion data computed using data 
in the instructional materials and student data object. 
20 The materials are represented in the ABI System by 

materials engine 604 and materials data 605. As previously 
described, the materials data comprises objects for display 
output or student input, sequencing logic controlling object 
display, and notations which when referenced by the 
25 sequencing logic during object presentation generate event 
messages for the agent. Materials data is advantageously 
grouped into "entries" comprising those objects, logic 
sections, and notations related to the display of a single 
screen or a few screens related to a single educational item. 
30 The materials data also preferably include a special header 
or initialization entry wiv;h important notations that 
communicate to the agent the educational paradigm adopted by 
these materials and can inform the agent of the control 
variables to which it is responsive. Such header or 
35 initialization messages are also preferably sent to the agent 
if the materials change their education paradigm. In a 
preferred embodiment, there is one common materials engine 
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604 which presents a plurality of materials data 605 la 
alternative embodiment, there can be specialized materials 
engines f or various classes or types of materials data 

5 data "oTlnf e " 9ine °" ° £ 

da.a 605. interprets the sequencing logic, displays objects 

as revested, and references embedded notations genera ing 

the requested event messages to the agent, standard 

facr ltles 0£ the OS and ES are used for object presentation 

and for routing any input to the materials. Input is 

IT"! aCC ° rdin9 t0 the seducing 

logic in a preferred embodiment, the agent controls the 
materials by setting shared global variables which the 
sequencing logic checks. Such shared global variables can be 
made available by OS interprocess communication ripe-, 
facilities, such as shared memory. The materials engine can 
be any program implementing these requirements, in particular 
« extension of available con.ercia! authoring co J s ^ 
engines, such as Macromedia's Authorware. 

in an alternative embodiment, the materials are not 

:;r at : an en3ine and * a ^ »» — ^ of 

one unitary component for each material, m this embodiment 
a process similar to compiling creates a single executable 
materials component. This invention is equally adaptable to 
otner implementations o, the materials that general agent 
» messages and are responsive to agent control according to the 
requirements outlined herein. 

Agent processing is divided into two consents, agent 
actxon processing SOS, which determines agent display actions 

30 whTr 56 t0 6VentS ' ^ ^ behaVi ° r P-essing 612 
the student, m a preferred embodiment, agent action 
processing is rule based and event driven. Rules in rules 

ll lTJ 10 a " eValUat6d USin9 Parameters both communicated 
xn event messages from the materials or the student and 

35 retrieved form the student data object. These rules propose 

candidate actions and then weigh and select a final set of 

agent actions from the candidates, which are communicated to 
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subsequent agent behavior processing 612 . Agent processing 
also sets global variables for materials sequencing and 
control. A side effect of this processing is the updating 
the student data object with information from the materials 
5 event message. 

Agent behavior processing 612 constructs an on-screen 
agent display based on the actions determined in agent action 
processing. In a preferred embodiment, this processing is 
based on behavior tables 613. Utterances, text or voice, and 

10 affects are selected from tables based on the determined 
final actions and refined with parameters included with the 
actions. The utterances and actions are sent to the selected 
agent persona object, which creates the agent display of the 
selected personae using the utterances and effect selected. 

15 Data is referenced and updated in the student data object by 
this processing, in particular fields reflecting the 
student's agent personalization choices, such as the desired 
personae, and fields reflecting recent agent behaviors. 

Also illustrated is communication materials 615. These 

20 materials manage and provide resources for various group 
activities, such as student tutoring, group work with 
particular materials, and group contests. 

5.4.2, The Executive Software 

25 The executive software {"ES") collects together a number 

of components which customize the operating system to the 
requirements of this invention and, if needed, also extend 
it. ES software implements common and special facilities. 
Exemplary common facilities include task control, 

30 communications, and I/O facilities; exemplary special 
facilities include a preferred animation facility and a 
client logon and startup facility. In a preferred 
implementation, ES software is built as frontends, wrappers, 
or extensions to available OS software components. For 

3 5 example, certain facilities can be implemented in C or C+ + 
and directly call OS interfaces, such as the Windows™ 
graphics device interface or its extensions. Other 
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facilities can be built- a* „i 

iBnl ° Ullt as cla sses using available 

implementation lanquaoe lihr-=.r-^c 

Java™ i o Y.r . llbrar ^s, such as the packages in 

^ ~,:t" iy - 

The task control facility manages the startup of svsten, 
components. First if t «*i-up or system 

10 student's aaent ran k • Indirectly, the 

ent s agent can be queried to determine if this 
activation is reasonable Nexr • 

manages the loadina Q , Pitted, this facility 

from 9 ^"•O software and data objects 

from server systems. Pi nally , task control ^« 

the component by making any necessary OS calls Ta s T 
15 also notifiPQ ^h a . cans. Task control 

Th co Z ° ther . SyStem events at task terminations 
The communications facility manages network 
communications and provides whatever client-server or file 
server protocols are not present in the OS <TT 

might include HTTP V2 . 0 with URL nL 
on ma . . wlcn URL nam e resolution. it 

20 mamtarns all necessary network sessions includinc" • 
with the three m aj or classes of servers sZlT2 f T^* 
servers, instructions! material server .' IT^^l^ 
software servers n a i- a *™ ^ ^ executable 
these server classes V i ™" lde ° n ° f 

25 servers C aTb o 7 ' aIterMtlv « -Potations these 

Finally this f ^ " SSVeral PhySiCal S6rVer S ^ Stems - 
anally, this facility handles all remote aC r P « 

information including requests for 2 , ! re ^ts for 

The I/O f .Hi>? re ^ uests downloading and uploading. 

The I/O facilities includes input and output displav 
handlers for object level display i/ 0 Th. hi 
30 receive object i Pwl display handlers 

audio and ate th ^ 9 " PhlC '' ^ — 

by the OS The i" hi:r° Wh3teVer int6rfaCe 18 SUPP ° rted 
me input handler receives low-level in™„- c * 

the OS input handlers and processes them .cco^,,^ th 

call the output handler to perform immediate feedback 
actions, such as hi gh li gh ti ng or dragging an o bj ect In 
other cases, it can pass x/0 event messages in I 
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format to the system component owning that screen object, for 
example, the on-screen agent. The output handler receives 
object presentation specifications, such as system owner, 
position, characteristics, size, any animation information, 
5 and whether the object is an input focus, then updates any 
necessary screen mapping, and generates OS requests to 
display the object. An example of such an object 
specification includes a selectable text field object with 
specified contents, perhaps scrollable, displayed by a 

10 particular instructional material. 

Exemplary specialized ES facilities are animation and 
client startup. It is preferable that the client system 
support animation, which is a connected and timed sequence of 
displays potentially calling on all display modalities 

15 available, and other timed presentations. Although this 

invention is adaptable to any suitable animation facility, a 
preferred facility presents a script based interface. 

The inputs to an animation facility are scripts which 
comprise object display commands, the timing of the object 

20 displays, an input specification of how any permitted user 
inputs are to be handled, and the types of events to be 
returned to the initiating system component. Examples of 
display events include user input causing script branching 
and termination or interruption. The animation facility 

25 receives a script from another system component, requests 
downloading of necessary display objects, for example from 
data snip libraries, interprets the timing specification, and 
sends request to the input and output handlers. 

The ES startup facility is described herein with respect 

30 to a student client to run on a network attached computer. 
Startup is similar for other client systems, including 
teacher and instructional designer clients. Each component 
of the ABI system must be downloaded from a server when 
needed, if as is typical, no component of the system is 

35 resident on a client prior to startup. The initial step 

involves the student accessing any client attached to an ABI 
network. Accessing can involve as little as powering-on the 
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computer and requesting startup of the svst™ „ 

Preferably, accessing by a fltu L, yStem ° n this ci ^nt. 

the server with the LsLm beginS * accessing 

standard intrane t man39er ' f ° r eX ^' «-ing a 

intranet browser which can be re.irion, 

" lni »«e s necessary eviction sessL^' ? ^ 
-th the system servers, an, then^ 
— 9« software, subtypes of the 
» 3 e„t software, and the scheduler/calendar " , 
The session manager presents the , \ software. 
« aepenaino on student pre'renl ITT "T* " " C °™ 
and receives input from th. \ " ^ da " ° b i e « 

-heaule/caiendar in cTope't " °' diSPl ^ 

-te^ines what ^JIZZTZZZ T^T ^ 
or can select from. Finally th. „ Presented with 

« are then downloaded and tne substan 

commences. Stance of the student session 

downloaded from a se" and"! "* ^ ^ 
» — y o bj ects, or datl X^'™ ZZSZT " ' ^ 
corresponding to the particular or* B / into ^oup S 

these linked groups can i i de ZlTT^ ' 
allow for a branching and interact t*™ °' ^ that 
Thus, one meta-resooL ~ Short -eta-response. 
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5.4.3. Syptifm Event Processing 

The ABI System on the student client, and on other 
clients, is advantageously organized to wait for student and 
timer input events and then to respond appropriately to these 
5 events. This is common current practice for constructing 
interactive systems. However, this invention is adaptable Co 
practices that might be developed in the future for 
constructing interactive systems as long as the principal 
system components and their mutual interactions as previously 
10 described can be represented in such a future practice. 

With reference to Fig. 7, this section sequentially 
describes the system processing of an input event and its 
consequences from initial input until the system waits for 
the next event. This description provides detail of how the 
15 system components communicate with each other and of how 
information and control flows through the system. More 
detailed description of the individual components appears in 
following sections. 

Fig. 7 conceptually illustrates control flow and event 
20 processing in a student client. Generally, at the left of 
this figure is relevant data,- in the center is processing 
steps,- and at the right are I/O events. As is apparent from 
the context, the arrows in this diagram represent either data 
references or control flow. The control flow is at times 
25 implemented by information bearing messages and at other 
times effected by standard ES and OS system calls. Further, 
this figure is directed to the processing of those components 
that interact with the student's agent. Those components, 
such as the session manager, which do not interact with the 
3 0 agent can be constructed according to standard interactive 
techniques (see, e.g., Petzold, 1996, Programming w<p h^«> 
33., Microsoft Press, Redmond WA) . 

Thus, at wait point 701 the system waits for the next 
student input or timer event. This wait commences at wait 
35 points 707, 713 and 717 after the system completes processing 
of the previous event, or possibly after system startup. 
Upon receiving an input event, ES I/O handlers 702 decide 
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whether the event represents an action without significance 
for other system components. For example, the event can be a 
student action causing only the highlighting of an object or 
a txmer event directing an animation. Such events can be 
5 entirely processed by the l/o handlers, and system waiting 
resumes at 701. if the event represents a student request 
for the materials engine 703 or a meta-request for the agent 
711, the I/O handlers format it appropriately and 
communicates it to the correct component owning that input 
10 object. If the event represents a time interval set by one 
of the materials engines or by the agent, it is formatted and 
passed to the requesting component, m the case of group 
work the event can be generated on a remote computer system 
and transmitted over a network link in order to reach tL 
15 local l/o handlers. 

The l/o handlers format event messages into the format 
expected by the component they are directed to. They include 
in each message an indication of the type of student action 
or time interval along with any input information from the 
20 action. One input event can generate several events 
messages . 

Herein, messages directed to materials engines 703 are 
considered first; messages directed to agent action 
processing 711 are considered second. Although Fig 7 

25 illustrates two materials engines, one engine or more than 
two engines are possible on a system from time to time 
Regardless of the number of materials engines present, event 
messages are communicated to the correct engine The 
materials engines 703 process a plurality of read-only 

30 materials data 704 representing instructional materials 
tools, and communication materials. As indicated by arrow 
725, these engines also access control information determined 
by agent action processing 711. This control information can 
be accessed in any convenient manner, such as by an exchange 
35 of messages or by referencing parameters stored in a shared 
memory region. This information controls the materials 
engines at educationally significant points during their 
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materials presentation. Optionally, materials engines 703 
can also access remote databases 705 and other remote 
resources available through the system network. The engines 
use these two sources, and optionally three sources, of input 
5 to cause presentation 706 to the student of, for example, an 
instructional lesson, a homework exercise, an instructional 
tool, or a joint educational contest. This presentation uses 
the I/O handler of the ES and OS to generate actual student 
display. 

10 As indicated by arrow 727, the materials engines also 

generate messages directed to agent action processing 711. 
At educationally significant points during materials 
presentation, notations in the materials cause the engines to 
format a message to the agent. These messages also include 

15 an indication of event type and relevant data, perhaps 
including timing data. One student input can generate 
several agent messages . At points which are not 
educationally significant, the materials can not generate 
agent messages. Thus, the system can wait again for the next 

20 student action at wait point 707 if the previous student 

action had either no input significance for the materials or 
agent or no educational significance for the agent. 

However, if messages have been sent to the agent, agent 
action processing 711 is activated. In the preferred but not 

25 limiting embodiment, this processing is table based. Rules 
from policy filter table 708 are evaluated in view of the 
data included in an incoming event message and in view of 
data on the student's past performance and the student's 
pedagogic model in student data object 712. For example, 

30 comparisons with student history are needed to determine 
relative performance. These rules propose candidate agent 
actions. Next, rules in decision weight table 709 and in 
selection criteria table 710 filter the proposed actions into 
a final list of agent actions. These final actions can cause 

3 5 the update of the information in student data object 712 with 
data on the student's current performance and behavior, as 
indicated by double headed arrow 728. They can also make 
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display ™ ~:^: bl ? acti °r can 

The lisr of «=■ , ^. -"sing visible agent behavior 

L a . 9 • Xt 13 P° ss ^le that no visible agent 
behavior is appropriate and that the list of disnf! • 
is emntv Tn t.h<„ display actions 

b empty. i n this case the system waits for furth^ ^ * 
input actions at wait point 713. C StUdSnt 

Agent behavior processing 716 processes fho r , 
10 of display actions communicated froT^ent » , St 
7U in view of utterance tables 714 dLl T? prOC — ia * 
7is. Preferences and the recor re * f gen! J^T^ 
-udent data object 712 in order to ZTrJTTcT^ " 

d is P i ay of the on . screen agent person 9 ae ne :; c \ c ° 

15 previous actions. The final , ^acting to the 

rinal actions select utterance 

r ::r: n r c i h r e compieted with ~ - 

selected T ^ diSplay behavi - table 

aCC ° rdln9 t0 the preferences use these 

complete utterances to generate display objects and „ 
20 animation scripts or aonl^c >, • v, ot5 3ects, and perhaps 

<= ... applets, which are sent to the I/O 

facilities of the ES for final . displav ? . 

Processing also updates the student data objecfw th 
information concerning this current behavior 

2S input^LYTrrV 11 P ° SSible PrOC6SSin9 ° f the ~ 
put action, the system now waits at wait point 717 for rho 

next student action or time interval. 

This section describes a preferred embodiment for th„ 
30 instructional materials, the student tools, and 

communications materials. ln this embodiment, the materials 
have a umf orm structure, being defined by materials 

which i S used by the materials engine r PP ; ; I ly data 

c^tain tools and the ^2^^^^^ ' 
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example, can be advantageously separately implemented as 
separate programs that themselves maintain the necessary 
agent interface. Such certain tools include a calculator, a 
dictionary, an encyclopedia, and group communications. In 
5 further alternative embodiments, each instructional material 
could be a separate program that also maintained the 
necessary agent interface. 

This section first describes the general structure of 
the instructional materials and then describes the tools 
10 typically available on an ABI system. 

5-5.1. The Instructional Materials 

In a preferred embodiment a common materials engine 
interprets specific materials data to perform instructional 

15 and tools functions. These are described with reference to 
instructional materials with adaptations needed for the other 
materials noted. Turning first to the materials data, in the 
preferred embodiment it includes three principal sections for 
presentation items, sequencing logic, and notations. The 

20 presentation items include whatever is displayed, preferably 
represented as display objects, which can be parametrized. 
These display objects can be packaged with the materials data 
or can be downloaded from a server on demand. The notations 
contain additional data related to the materials display. 

25 These include, for example, prerequisites, links to related 
material, expected student performance, and help and hints. 
The notations are preferably generated from templates 
referencing parameters from the materials data and student 
performance inputs. The materials engine uses notations to 

30 generate messages to the agent, which comprise one part of 
the agent interface. The sequencing logic is executable or 
interpreted code that animates the particular materials. It 
references all data in the particular materials to cause the 
ordered display of the presentation items and to send 

35 messages to the agent according to the notations. 

The materials data is advantageously grouped into 
entries, each entry representing a minimum item of 
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presentation, „ hich oan , however involve 
These entries are preferably speciaUled „ s ~ 
header or inrtialUation entry ana the other entries. Table 
2A illustrates a typical materials header entry which is the 
. first entry processed when the materials are initialized 

TABLE 2A: HEADER ENTRY 



10 



15 



20 







Instructional 
[ Header Entry 






Presentation 
items 


Sequencing Logic 


Notations 




Global variables; 
Global functions; 
Local variables; 
Local functions; 


Prerequisites for 
materials; 
Reference to other 
materials or standard 
texts . 

Agent initializing 
information including 

- meaning of global 
variables ; 

- education paradigm 
adopted 

- available helps, tools 



Thxs header has several particular features. The sequencing 
logxc for the header frame consists largely of definLons of 
variables and functions. The global variables are shared 
" that^H SyStem COmP ° nentS and the control variables 

nd t I 396 " that ^ ^ Terences 

and tests m order to be guided by agent control. These 

variables, which comprise another part of the agent 

interface, for example, can control the time pacing of 

30 instruction, the new concept seeding rate, the density of new 
example,, the time pacing, or the difficulty of 
discriminations. Local variables are available to the 
sequencing logic during materials processing. Global 
functions are those global system functions that can be 

35 "lied by the sequencing logic. Also, dictionary lookup 
spell checking, or encyclopedia lookup can'be globally ' 
implemented and shared. m a Windows* environment the 
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global functions can be DLLs. Finally, local functions are 
available locally to the sequencing logic. For example, in a 
scientific calculator tool, computations can be local 
functions. For a further example, in instructional materials 
5 the local functions are available for checking user inputs 
for correctness, scoring quizzes, and so forth. 

The notations in the header entry generate materials 
initialization messages to the agent. These messages inform 
the agent about these materials, about what global variables 

10 they respond to, about what helps, hints, and tools are 
useful, and importantly about the educational paradigm the 
materials use. Preferably, information about this paradigm 
is structured according to the instructional materials 
interface standard. See infra. The notations can also 

15 contain additional information, such as prerequisites for the 
whole materials and references to other materials and texts. 
Finally, the presentation item can be, for example, an 
introductory screen. 

Table 2B illustrates a typical entry which is processed 

20 during regular materials presentation. 



TABLE 2B: MATERIALS ENTRY 



25 



30 



Instructional 
Entry N 






Presentation 
items 


Sequencing Logic 


Notations 


Display 
objects, 
possibly 
including 
those for 
text, 
graphics , 
sound, 

animation, cr 
video 


Interpreted or 
executed control 
statements 
referencing 
variables, 
notations, and 
any student 
input , and 
causing display 
of display 
objects and 
execution of 
notations 


Prerequisites for frame; 
Expected timing, and 
difficulty; 

Relation to other frames; 
Links to other materials 
and sources . 
Changes to agent 
initializing information; 
Student performance data; 
Student error data 
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For thzs entry, the presentation items are those for the 

" tB ' / iSPlay - ^ SeqUenCin9 l0 ^ ic ^is display 

xn vxew of the variables and other information in the 

5 TellT": ^ StUd6nt inPUt " Finally ' the 

set a ^ m : SSa96S reP ° rting Cha ^ es i» -y Parameters 
"ro^ lni ri tial " atl ° n ' StUdent Performance data, student 

S ' ° th6r ed -ationally significant information The 

notatxons can also contain information specific to this 
frame, such as expected difficulty and timing 
10 Turning now to the materials engine, its first 

processing step is to request the executive software to 

ZTrZVll reqU6Sted matSrialS fr ° m ^ instructional 

materxals server. it next processes the header entry i ink , 

to global variables and functions, and sends ini n za ^ 
» event messages tQ the agfint when presenfcation ' 

r:;: p f r ::: e or r iis r execution the ^ *» 

frlmes r , ' " ^ ^ Pr ° CeedS C ° other 

frames as directed by the sequencing logic. Lastly when the 

materxals processing ends, any termination message!^ 
20 dxrected by the notations are sent to the agent and the 
materxals are deleted from the client. 

in a preferred embodiment, the materials are all 
implemented similarly. Mos t differences between the 

25 ZTnlT' matSrialS ' t0 ° lB ' ^ materials 
are xn the presentation items and the sequencing logic 

including different global and local entities. A n ma ' terials 

are expected to have notations f or generating agent messes 

t at record materials initiation and terminal 1 

performance and errors. Preferably this information is 

30 reported in a standardized manner according to an 

xnstructional materials ..nterface standard. See infra. 

5.5.2. The Tor>1 ;a 
The ABI system is adaptable to a wide range of necessarv 
35 and optional student and teacher tools tailored to thl 
students and the courses of instructions: m the case of 
elementary education, the following preferable tools include 
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certain general tools and the communication, or joint work, 
materials. In the case of commercial education, other tools 
can be preferable. The schedule/calendar tool participates 
in permitting access to materials according to student 
5 schedule and is preferably found in all embodiments. 

The discussion in the section is directed to an 
implementation for elementary education. It is not limiting 
in the tools that can be used in an implementation. 

10 5.5,2.1. The General Tools 

The general tools are preferably present in a range of 
forms selected according to data in the student data object, 
including grade level, and teacher permissions. One general 
tool is a calculator, which can have forms varying from a 

15 simple four function calculator to a complex graphing 

calculator. Other general tools include language tools, such 
as a spelling checker, a thesaurus, a word pronouncer, an 
encyclopedia, and a dictionary. Different levels of each 
language tool can be provided suitable for different grade 

20 levels. In an alternative embodiment, the language tools can 
be integrated with a "starfish" tool, which allows the user 
to place the center of a "starfish" over a particular word, 
and then, by selecting the appropriate arm of the starfish to 
obtain the definition, a pronunciation of the word, a rhyming 

25 word, a synonym, or an antonym. Another general tool is a 
word processor, perhaps with a drawing mode which can be 
provided as a multi- level set of writing and drawing tools. 
The writing and drawing capabilities available to a student 
are selectable by the student or teacher. A last general 

30 tool is a link-maker, which offers exercises in various types 
of memorization, such as paired associates, serial learning, 
ordered serial learning, and mnemonics. Finally, additional 
tools can be added to an implementation to meet specific 
educational needs. For example for geography lessons a map 

35 tool can be added. For student projects, an encyclopedia 
tool and a network search tool can be added. The Study 
Buddies™ can provide instruction in operation of the tools 
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such as the use of keywords and operators Sn ^- i ■ , 
can be added for r nmm0 v • , orators. Specialized tools 

aaaed for commercial or industrial training. 

5.5.2.2. File Sv„«- Bn 

-mentary education /"n rL^^r^ 
provided file system can be . disadvantage Cole T" 

and access to the ^ f ^s ^ ^ °' 

is advantageously implemented s ^t oT^JT^" ^ 
and not as a material, with that Zll ° 
iconic file represents- Cementation, customized 

e re P r esentations are manaqed as mt^ „* ^ 
15 area of the display. P ° f the s y stem 

One embodiment of such a fn a „ ■. 
four level fii* , , yStem t001 P^ents a 

our level file system organized into shelves of hnnt 
of which has sections n, c . sneiv es of books, each 

* sections, the sections having pages a bnm<- 
representing a directory of files is " ! 

20 user selects its icon The stulnr * 

student navigates around rh*» 
dlrec ory boo* by seating tabs or by m ovi ng page by p * 
by select lng „e„tpa g e, lastpage , or ot „ er * 
student interacts with selected pages or f ,,„. , ™* 
P-iMy create ne„ P a g es M LctLT e T.T 

Mere L T Prespecified bocks is adequate 

"ore advanced students can be given pension to create and 

p™::^^ttr:::^rbt k an v; cut r paBte 

30 -a i pre f era bl y presented ^ iL^ ^ 

it. For example, user-create . . F cess 

with the M nr/ created text or giaphics pages appear 

with the word processor active. Homework and instruction 
pages appear with the appropriate materials. 



35 
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5.5.2.3. The Scheduler /Calendar Tool 

The schedule/calendar is an important tool and is 
preferably always present. It is accessed when the ABI 
system initiates materials to verify the student is permitted 
5 and scheduled for this material, and also invoked when the 
•system terminates materials to schedule new materials. It is 
accessed as a global function by the agent in response to a 
meta-reguest from the student seeking scheduling assistance. 
Further, it can be directly accessed by the student using the 
10 calendar tool icon appearing on the student desktop. When 
accessed, this tool displays a calendar book to the student, 
viewable in several ways . 

This section describes, first, the schedule/calendar 
data, and second, the schedule/calendar processing. The 
15 schedule/calendar data is a subtype of and contained in the 
student data object. Generally, this data includes the 
following fields for each scheduled student activity: 

Name of scheduled activity and optionally an identifying 
message; 
20 Activity priority; 

Deadline date and time, or a definition of a perpetual 
activity, which has periodic requirements but no 
completion date; 
Link to material for the activity which in turn can 
25 specify activity completion criteria; for activity 

of a single student this is typically a particular 
instructional material; for group work activity a 
list of the students for the group and other 
communication information can also be in the 
30 calendar entry; 

Activity characteristics, for example whether this was 
entered by the student or teacher and whether this 
is to be marked complete by the student or system; 
Activity status, completion status and submission status 
35 of any required reports. 

This exemplary data, sufficient to define a scheduled 
activity, can alternatively be entered by the teacher or by 
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ToJ tU TT' " 6ntered ^ ^ t6aCher ' ifc «» be Protect- 
from modification by the student. Data entry is preferably 

assisted by a teacher tool with a suitable screen format 
Optional activities can be entered by the student if the' 
5 teacher permits. 

The schedule/calendar tool can be directed by the 
teacher to permit the student a range of scheduling 
initiatives. These initiatives can range fro m permitting the 
student freedom to schedule materials as desired to requiring 
10 the schedule/calendar tool to enforce a fixed order of 
student activities. m a preferred embodiment this is 
accomplished by the teacher's specifying initiative 
parameters including a deadline date/time, D, and a criterion 
priority, p, in the student data object The 
15 schedule/calendar then schedules automatically all tasks with 
deadline less than or equal to D and with priority level 

ITITST, " SqUal ^ ^ ^ Varyin9 D and P -spect 
to scheduled tasks the teacher can achieve the specified 
range of initiatives. 
20 Schedule/calendar processing can be invoked by the 

executive software, by the student, or by the agent As 
previously described, at the beginning of a student ' session 
or after termination of the use of a particular material the 
executive software invokes schedule/calendar tool first to 
25 mark the terminated task complete, and then, to reference the 
calendar data in view of the initiative parameters to find 
activities requiring scheduling, if this tool finds only one 
activity in the calendar requiring scheduling, this required 
activity is initiated, if raultiple required activ±ties 
30 found, the tool can, alternately, initiate the required 

activity of highest priority or allow the student a choice of 
which required task to initiate. If there are no required 
activities, the schedule/calendar tool allows student 
selection of the next task via the session manager 
35 The schedule/calendar tool can be invoked by the student 

by selecting the schedule/calendar icon in the system area of 
the display. When so invoked, the tool displays the student 
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calendar and scheduled activities in various formats from a 
monthly overview to details of individual activities. If 
permitted, the student can enter optional items or mark items 
complete . 

5 The schedule/calendar tool can also be invoked by the 

agent when it receives a student meta-request of the type 
"What do I do next?" The agent retrieves the required and 
scheduled activities from this tool and also determines an 
expected time to complete each task based on student 
10 performance from the student data object and the average time 
required for each task from the materials header. In view of 
this combined information, the agent can present to the 
student an ordered list of activities scheduled according to 
their expected time to complete. 

15 

5.5.2.4. Communications Tools and Group Work Materials 

In a preferred embodiment, the ABI system includes 
communication, or group work, materials integrated with the 
remainder of the system. As for other tools and materials, 

20 access to communications materials is granted by the 
scheduler/calendar tool . Communication work groups are 
assigned and scheduled in students' calendars with calendar 
entries preferably including the group members names and 
other communication parameters. When these materials are 

25 activated by the scheduler, the communication group is begun. 
Alternatively, students can spontaneously request the 
formation of a communication group by the selection of a 
communication material. The scheduler/calendar tool can 
permit group activation if the students have no other 

30 required activities. Alternatively, each particular 

communication material can also have specific access controls 
preferably set by the teacher that control the types of 
communication permitted and with whom the communications is 
permitted. 

35 In manner similar to other tools and materials, the 

communication materials have an agent interface. Upon 
activation, they send initialization event messages to the 
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agent specifying the global control variables they will be 
sens t educational paradigm adopted, and available 

hints, helps, an d other communication parameters i„ 
alternative embodiments, the instructional materials 
5 interface standards include special categories for 

communication based work that enable the agent to control 
these materials with specificity. Du ring communication work 
hese materials generate event messages at educationally 
significant points. y 

10 Thereby, communication aerials are fully integrated 

IZ " *" e "* 0dlment - F "" h -' in • embodiment 
«ication macerials are ^ leinentea in 

to other aerials. Pirst, Mch comilnication . t ^"^ , 

15 Lte " C °" UniC " i0 " »P^«= chat communist on 

15 material or form of group work. The communication task 
manages the network interface for that particular type of 
communication or group activity by using the net „ or ^ 
protocols provided by the OS and ES. and provides its 
communication functions as global functions for access 

available to the student in a manner similar to other 
materials through particular materials data that includes 
presentation items, sequencing lo gi c referencing these global 
communication functions, and notations generating event 

35 messages for the agent. m an alternative embodiment, the 
communicate materials can be programs., independent of the 
materials engine and perhaps part of the associated 
communication task, which internally generate the necessary 
agent event messages, in either embodiment, communication 

30 materials tasks can be written either in the Ml 
implementation languages, or in a special purpo-e 
communication scripting language. 

The particular communication materials in a preferred 
embodiment provide forms of group work or communication 

for joint work on materials, and structured joint work such 
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as contests with rules. Each of these forms of group work or 
communication is described in the remainder of this section. 

A first form of group work implemented by communication 
materials is E-mail and newsgroups. These are useful for 
5 teachers to send information to their classes, such as 

schedule and materials changes and to communicate with absent 
students. Teachers can also exchange information with each 
other or obtain help for system and other issues. Students 
can use this form to obtain help and advice, especially from 

10 remote sources, communicate with their teachers, and share 
work or interests with other students. E-mail and newsgroups 
are easily incorporated as previously discussed. Then ' 
materials data is authored that grants access to these 
functions and generates appropriate agent event messages. 

15 Student linking is another form of group work 

implemented by communication materials. Students in session 
at separate clients can link together for various exemplary 
activities including simply talking with each other by voice 
or text or for joint work on a particular material in which 

20 the students have either similar roles, as in developing a 
document using a word processor, or different roles, as in a 
simulation or game. Another activity of linked students 
includes group activities, in which position of participants 
within a virtual environment determines activity and role 

25 within activity. A final exemplary activity for linking 

student groups is moderated activity, in which participation 
is controlled by a special coordinating task that perhaps 
executes on a server system. An example of this latter 
activity is a spelling bee which is described in more detail 

30 subsequently . 

In the preferred embodiment, student linking includes 
the following steps. The first step is identification of 
other students with whom a given student can link. The group 
can be defined by the teacher in the schedule/calendar entry 

35 for this activity, or alternatively, in a communication 
access control particular to this linked work activity. 
Second, links must be established between the students to be 
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linked. These links can be to a single server communication 
materials task that receives and distributes messages 
Third, local and global actions must be determined. Local 
actions are those that result in output visible only to the 
5 user taking the action. Global actions are those that result 
in output visible to all the students in the linked group - 
These global actions can include communicating each student's 
input to all linked students, sharing information among all 
linked students, jointly creating information by students in 
10 the linked group, and storing jointly created information 
The fourth step in linking is orderly disconnection from the 
linked group . 

Linking can be implemented in alternative fashions m 
all implementations the communications tools and materials 
15 for linked activities are integrated with the other 

components of an ABI system in one of the ways previously 
described. A simple implementation is to provide on each 
student's screen an icon and a message area for each linked 
student. Alternatively, one or more shared materials areas 
20 can be provided. Communication can be distributed through a 
single server task to which all linked students connect A 
more advanced implementation of linking employs software 
packages similar to multi-user dungeons ("MUDs"), which 
contain a collection of virtual shared traversable spaces 
25 (called "rooms") in which users can interact either with each 
other or with elements within the room. MUDs are especially 
preferable for student group work in which different students 
have different roles with access to different system 
capabilities. For example, one student can be a recorder 
30 having write-access to the group's notebook, while another 
can be the database expert, having access to a body of 
relevant data. MUDs are also be useful for teachers, 
communicating with each other within 'rooms' each set aside 
for a specific topic and forming a dynamic newsgroup. 
35 Another form of group work implemented by communication 

materials is structured linking. Important examples of 
structured linking in which the students have different or 
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structured roles are educational contests. Exemplary of such 
contests is a spelling bee, an exemplary embodiment of which 
is described herein. In this embodiment, the spelling bee is 
managed by a server communication materials task, called the 
5 server spelling bee task, which preferably is executed on a 
server system, which communicates with local spelling bee 
tasks on the involved students' client system. Generally, 
the server spelling bee task initiates a spelling bee by 
accumulating connections with students local spelling bee 
10 tasks, then mediates the spelling bee by controlling student 
actions, and provides for orderly termination of the spelling 
bee. The local spelling bee tasks provide the communication 
functions accessed or required by the spelling bee materials 
data, which are scheduled or selected to invoke spelling bee 
15 participation, on the client systems. These materials also 
send event messages to the agent and are controlled by the 
student's agent. Alternately, the local spelling bee tasks 
can be programmed to communicate with the agent and perform 
the spelling bee without materials data. 
20 in more detail, the spelling bee tasks carry out the 

following. steps. The spelling bee server task is started at 
a teacher's request or automatically by the system. A list 
of eligible students can be selected by the teacher or 
automatically determined by the server task based on data in 
25 the student data objects, including class membership and a 
stated interest in spelling bees. The spelling bee activity 
can be scheduled for each individual student by the teacher 
or selected by the student. If a student desires to enter 
the spelling bee, the local spelling bee task is started by 
3 0 the ES and a message is returned to the server. No response 
within a specified amount of time is taken as indicating a 
desire not to join. if a student's data object shows that 
this is the first time in a spelling bee, an instructional 
and warm-up sequence can be authorized for the student by the 
3 5 agent in the server task. If enough eligible students join 
the spelling bee, the server task continues, otherwise it 
sends a termination message to those who have joined and 
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th TT' l0Cal SPelUn9 bM task space on 

the student display indicating the other players J J." 

inputs. „e*t the spelling be e begins, and the server task 

5 list and the name o, user to spell the word. Each local "ask 
echoes the word sent, preferably by revesting the on lee" 

J" ™ " UCteranC " " lth M 'PP-priate 

affect. The spelling bee materials inform the student's 
local agent of the student's progress and performance in the 
10 spellmg bee materials. The server task accepts in^t trT 
designated user's local task and broadcasts it. ThT erveT 

ask lodges ana reports on correctness of completed reuses 
and if lncorrect , eliminates the stu(jent from £urt SP °" Ses 

spellmg requests. As student leave the spelling bee the 

oHL""" ^ nOCi£ied ^ SSndS local tasks 

Of ™n 9 players in order to update their workspaces 

in each case the student preferences for further speUing' 
bees are checked and preference dai-a <„ . ^ ^ 

object is updated. „Ln 0 J ZT 111 „ f^T^" 
a. ^Uing bee task terminates the game an! ^T^r" 

The agent is an important component of this invention 

25 h n e r ther describea in this eecti - «««^ " 

» the student data object, the agent interface, agent 
processing, and agent adaptivity. The agent comprises 
student data object which contains data on the student's 
performance on the various materials and data on the 
student's pedagogic model. The student data object is 

example, for report generation and for student scheduling 

alt „ COn,PODentE Pr6£erably ^ - to the 

agent in order that the agent can control the materials and 
gu.de the student in , uni£onn _ pr ^ 

35 deo mto two pnases, agent action processing and agent 
behavior processing. Fin ally, agent adaptivity in the 
preferred and in alternative embodiments is described 
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5.6.1. Student Data Object 

One student data object is created for each student in 
the ABI system and is the only permanent repository of data 
concerning that student. The student data comprises fixed 
5 daca defining the student as well as evolving data describing 
-he student's interaction with the system, the latter 
-ncluding current and past performance and data defining the 
agent's view of the student. The student data object is 
stored on the server system and is the source on the server 

10 system for all teacher and administrative reports concerning 
that student. Elements of the student data object are 
fetched to a client system as required once its associated 
student logs on to that client and on that client serves to 
control the agent and provide for agent adaptivity. 

15 Figs. 10A, 10B and 11 illustrate the structure and 

operation of the student data object. Figs. 10A and 10B 
conceptually illustrates an exemplary structure for student 
data object 1101. It is an object comprising structured 
student data 1102 and methods 1103 for accessing and updating 

20 the student data. Student data is divided into global data 
1104, materials related data 1105, including tool related 
data 1106, current lesson data 1107, and log data 1108. 
Global data, that is all items meaningful across all ABI 
materials, includes such subtypes as system data, agent 

25 behavior preference data 1109, agent student model data 1110, 
and schedule data 1111. System data includes student 
identifiers, student passwords, access privileges, grade 
levels, class registrations, etc. Agent behavior preference 
data 1109 relates to the multi-modal behaviors generated by 

30 the agent and includes student defined preferences for these 
behaviors as well as a summary of past agent behaviors. 
Student preferences can include options relating to agent 
visual appearance - species, gender, dress, or perhaps, no 
visual appearance - and similar options relating to audio 

35 behavior and text production. The summary of past agent 
behaviors is used to aid in the selection of reasonably 
varied future multi-modal behaviors. Agent student model 
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data mo includes items modeling the student's persistent 
behavior which the agent uses to individualize its 

material retention rate, hint effectiveness, and preferred 
5 rewards. Updates to this data object made as the Z c ent 

UTTT StUdent Pr ° Vide £ ° r — * adaptiv ty to 

the student. Finally, schedule data lln relates to 

assignments scheduled by the teacher and unchangeabie by the 
tudent and optional items scheduled by the student dL 

1™ S r edUle " n *» -inder 

alarms, and priorities. 

Each course in which the student is currently enrolled 
has a separate materials related data area H05 f or the 
materials providing instruction in this course. In an 

c^lT.r"^ 6 "' data SUbtyPe ™ and 

include oh USUally ^ SCh ° 01 SySte *' 

include ob 3 ect a ves and standards the student must meet in the 

particular course, milestone data establishing obje v 

already met by the student, data relating to the tudent s 

20 progress » the materials, data relating to the student'suse 
of tools in the materials, and performance data. Progress 
data deludes data necessary for the student to leave the 
materials and resume the materials at the prior point 
Performance data m 2 relates to student's performance over 

25 several i essons in che materials and _ ^ 

performance, weighted moving averages of performance 
patterns of performance, use of hints, use of retries and 
needed remediation. Using such performance data, for' 
example, means and weighted moving averages, permits the 
30 agent to determine whether student performance is improve 
or dedxnxng. Tool data H06 contains essentially similar" 
bu more abbreviated data about use of system t J s ^ 
the calculator, dictionary, and word processor. This data 
can only include milestones and performance information 

The status of each lesson presented by instructional 
materials is accumulated in current lesson data li 07 This 
subtype is created upon lesson initiation and deleted upon 



WO 97/44767 PCMJS97/08687 



lesson completion. In an exemplary embodiment, it includes 
short term measures of performance - such as error rates, 
weighted moving averages of error rates, and the use of hints 
and retries - short term measures of time latency - such as 
5 time to complete lesson segments and weighted moving averages 
of such times - work areas in which the agent can store 
information particular to the instructional materials - such 
as parameters to use in forming multimedia presentations - 
and lesson coaching parameters 1113. The lesson coaching 
10 parameters are used by the agent to provide feedback to the 
instructional materials so that their presentation can be 
individualized according to student performance. These ' 
parameters are governed by the instructional modalities 
employed by the particular instructional materials and can 
15 include values such as the seeding rate of new concepts, time 
pacing of the presentation, the density of examples, and the 
ratio of reinforcement . 

Finally, the student data object has links to student 
log 1108. The log stores all messages input to agent 
20 processing and all actions from agent behavior processing. 
It can be used to create detailed audits of student behavior 
and system responses, which can be of interest to 
instructional designers, in order to improve instructional 
materials, and to educational researchers, in order to 
25 develop new modalities of instruction . With its carefully 
partitioned and functionally defined interfaces, the ABI 
system is easily adaptable to new modalities of instruction 
as well as to merely installing new materials. 

The student data object also includes one or more data 
30 updating methods and one or more data accessing methods. 
Exemplary updating method 1114 includes two components, 
triggering event type 1115 and action list 1116. When a 
component of agent processing updates the student data 
object, it sends a message to the object including an update 
35 event type and a list of relevant parameters. The updating 
methods are searched to find one with a triggering event type 
which matches the event type of the update event. The one or 
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more methods having matching event types gre ^ 
performing all the actions in the included action list using 
he parameters in the update event message. Each action uses 
the Buppned parameters to update data elements in the 
5 student data object. A plurality of actions is generally 
associated with each method because it is typical for one 
event to cause changes in several student model data 
elements. For example, an update event related to the use of 
a tool causes changes in the relevant tool data subtype as 
10 well as the associated instructional material subtypf 
Further, when a lesson is completed, the methods executed 
approprxately summarize student data from the current lesson 
subtype into al! the percent data subtypes. At this point 
13 m'oTrr" SUCh 93 ' hint e " eCti — ^ th. agent tu'd n ' 

paramet ^ ^ ' hint ^ectiveness 

parameter 1B accessed for uses including agent decision 

I III 7Zr\TT r report 9eneration - one of the 

assocxated wxth the event type ■ end of lesson , es 
parameter -hint effectiveness' in the following Lemplary 
20 manner. The performance parameter 1120 'use of hints' Z 
accessed to determine if update is required. if hints were 
provxded to the student, the 'current lesson log- ll08 is 
used xn calculating the two components of the parameter 'hint 
effectxveness' - -nint effectiveness-before-first-try' and 

ea-.h hold an array of values which are used to compute a 
weighted moving average. In an alternative embodiment, a 
more complex formulation of this parameter can be used to 

30 TT* 5 m ° re detaUed analySlS ° f Mnt -"ectiveness - each 
of xts two components can be separated into subcomponents 
corresponding to subject area, hint types and other 
xnstructionally useful measures. it is further preferable, 
for data elements comparing a particular student to class 
peers according to various measures be entered into the 

35 student data object. This can be done by executing 

appropriate update methods on the server system where such 
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comparison data is available from reports generated from the 
student database including all the student data objects. 

Fig. 11 is an exemplary illustration of how a typical 
student action update event updates the student data object. 
5 Other update events include agent action update events. 
Update message 1201 is sent 1206 to the student data object 
from a component of agent processing to cause student model 
update. The update event type is M exercise_done" and 
exemplary associated parameters are as indicated at 1202. In 
10 this case method 1203 with triggering event type 

"exercise_done» is executed by performing the four associated 
actions. The first action updates relevant data elements in 
tool data subtype 1106. The second action updates the log 
1108, as is done for all events. The third action updates 
15 timing latency data elements 1205 in current lesson subtype 
1107. Finally, the fourth action updates student performance 
data elements 1204 in current lesson subtype 1107. 

Any component of the ABI system needing to determine the 
value of particular data element in the student data object 
20 does so by sending an inquiry message to the student data 
object requesting the desired data element. The inquiry 
method for that data element retrieves and then returns the 
desired value. Such inquiries are typically made by the 
agent on the student client system and by inquiry and report 
25 generating programs on the server systems. 

5-6.2. The Aae nt /Materia Is Interface 
The structure of the interface between the agent and the 
materials is important in the ABI system. It permits a 

30 single agent to control a wide range of materials through 

which it guides a single student. The agent achieves this by 
advantageously maintaining a model of the student's pedagogic 
characteristics, which it references in diverse situations to 
determine its actions. This section, first, describes the 

35 general procedural structure of this interface, and second, 
describes the preferred model for the content of the 
interface. This preferred model is structured according to 
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■™sT ruccional " aterU1 interface standard - 



Communxcations between the agent and the materials is 
b^rect.onal. Events containing palters are sent to the 
» agent by the materials at educationally significant 

occurrences. In response, the agent sets global parameters 
controlling the materials and returns usages con fi Jng 
actions proposed by the materials. In circumstances in which 
he materials needs to coordinate dismays with the agenf U 
10 communicates synchronously with the agent. Por example when 
he student requests help or a hint, the materials can need 
to synchronously obtain the agent's permission to otter the 
-elp or hint, m other circumstances, the materials can 
asynchronously send informational messages to the agent 
» u=h asynchronous agent input and possible output can give 
the system the appearance of spontaneity 

convent a9enC/mat « ials in «^« can be implemented in any 
consent manner in a given OS. Por example, it can be 

20 calls to I T meSEa9ln9 ' Shar6d 

20 calls to a socket interface, or other technology 

The global parameters set by the agent and which control 
the materials are preferably state variables that the 
materials sequencing logic references in order to make 

25 of U s C ta t tTv lly ^' 9nifiCant ^cisions. The meanings 

25 of state variables to which a particular material is 

sensitive can be established at materials initialization 
acc„rd lng Bpecitications in m header maceriais 

Examples of such variables range from simple flags, such as 

30 mo C ° h ntr0llin9 thS •""•Mlity of helps and hints, to 
30 more sophisticated parameters, such as those controlling the 
rate of new concept introduction, the density of examples or 
the speed of discrimination exercises. ™»<Pl«». or 

Communications from the materials to the agent are 

35 IZTT** ^ n0ta "° nS " materU1S da "' * "Nation 
includes an event type, parameters associated with the event 

and the condition under which the event is constructed and ' 

sent. Notations are activated when they are encountered in 
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sequencing logic in the materials data. There are two 
classes of notations, "point-wise" notations and "toggled" 
notations. Point-wise notations are evaluated exactly once 
when they are activated. Toggled notations are evaluated at 
5 each input event following their activation until they are 
inactivated by a later notation. 

Notations vary according to the materials. Some 
materials, such as simple e-mail, can contain no notations. 
Tool materials can contain notations indicating only correct 

10 or incorrect use of the tool. Most instructional materials 
data contain several types of notations. Generally , the 
events generated by these notations send information similar 
to the following: number of retries, measures of rate in 
fluency drills, measures of performance such as percent 

15 correct and the number of tries, partition of exercise items 
into similarity sets, and measures of the relative difficulty 
of items. 

Table 2C illustrates exemplary types of notations 
generated by typical instructional materials. 



25 
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TABLE 2C: TOPES OF NOTATIONS 



10 



NOTATION TYPE 



TYPICAL PARAMETERS 




Partial 
response 



15 




20 



Full response 



Help/hint 
request 



Point 



At partial answer 
for a requested 
input item 



At complete answer 
for a requesting 
input item 



Educational paradigm used; 
meanings of state variables 

Correctness of user input 
time required 



Name of exercise, 
difficulty, related 
exercises; correctness of 
user input, time reouirpH 



At an error 



Screen location that agent 
can highlight or move to 




Inappropriate 
input 



Inappropriate 
inputs from student 



25 



Number and type of 
inappropriate inputs 



30 



35 



5 - 6 ' 2 - 1 * The Ag CT t /Materia T^. ace Rtmrt « m ^ 

in order that the student's agent can act generally to 
proves student 9ultele . Md control J 

a manner individualized to the student's pedagogic 
characteristics, it is preferable that an embodiment of this 
invention use an instructional materials interface standard 

that the detailed content generated by the notations and 
passed in messages to the agent be structured in a standard 
fashion according to the particular educational paradigm 
adopted by the materials and independent of the particular 
content of the materials. Further, it is pre ferable that the 
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materials adopt one of a limited and defined set of 
educational paradigms contemplated in the standard. Finally, 
it is further preferable that the rules referenced by the 
agent in its associated processing tables and performance 
5 data in the student data object be similarly structured. 

Thereby, IMIS provides the agent with a materials independent 
view of the student. 

. IMIS is not limited to a particular set of educational 
paradigms. Any standard set or sets of paradigms appropriate 

10 to the intended students can be adopted for the interface 
standard. It is preferable that the standards adopted be 
based on principles of educational psychology and sound ' 
educational practice. In the following, this invention is 
described according to an IMIS appropriate for elementary 

15 education. The paradigms described below are not limiting. 

Exemplary educational paradigms 

Exemplary educational paradigms, also known as modes of 
instruction, are listed in Table 3. 

20 

TABLE 3 : EXEMPLARY EDUCATIONAL PARADIGMS 



Interactive tutorial 

Fluency exercise 

Paired association exercise 

Discrimination formation exercis e 
Simulation exercise 



Each of these educational paradigms is preferably 
handled differently by the agent in response to differing 
descriptive information and student performance data. For 
example, a sequence of correct responses in a fluency 
exercise is expected. On the other hand, a sequence of 
correct responses in a paired associates exercise can be 
worth while for the agent to comment on. 
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Exemplary par adigm e ia M <«..^^ 

The exemplary embodiment of imis standardizes these 
educational paradigms according to three pieces of 

5 f^rT'lT inStrUCti ° nal C — - tH. instructional 

format, and most specifically, the subject area. Materials 

z : r~> preferabiy specify ai1 *~ - 

agent flexibility, although the ABI system is adaptable to 
P elf 3 Id" 9 S SPSCifyin9 ^ ^ « — • - -ne ar 

io IZTZ gent actions are independent of the —^-i 

in.t/T" 8 '™" 10 " 31 C ° nteXt 18 thS S P 6Cific cf 

LaLle C "f 9 PreS6nted C ° StUd6nt * the 
Examples of instructional contexts are: 

" TABLE 4 : EXEMPLARY INSTRUCTIONAL CONTEXTS 



Prerequisite 
Test 
Review 

Pretest 

20 New material 

Introduction 

Discrimination 
Review 
Practice 

Fluency exercise 
Review 

25 Unit mastery test 



Within each instructional context, materials can adopt 
instructional formats, the second component of the IMIS 
specification . Examples of instructional formats are- 

30 



35 
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TABLE 5: EXEMPLARY INSTRUCTIONAL FORMATS 



Multiple Choice 
Fill-in-the-blank 
Unprompted 

5 Selected from list 

Paired Associates 

By letter 

By dragging 

By matching 
Computations 
Simulations 

10 Identify parts of figures or text 

Simulation games 



Finally, student performance should preferably be stored 
relative to the subject area being worked on, as necessary 
15 for course level reporting. Thus, the third component of the 
exemplary IMIS is the subject area, such as mathematics or 
reading . 

Exemplary standardization of agent data 

20 IMIS standardizes system tables and data relevant to 

agent action processing according to this triple of 
information -- instructional context, instructional format, 
subject area which characterize the education paradigm 
adopted by the materials. The following description 

25 discusses standardization of notations in the materials, 
standardization of data in the student data object, and 
standardization of the agent action processing tables. 

Notations in the materials are standardized according to 
IMIS as follows. For a lesson, exercise, or item, each 

30 material adopts one of the standard set of educational 
paradigms, or modes of instruction. The parameters to be 
passed to the agent in an event message are determined by the 
instructional context and each instructional format of this 
paradigm. 

35 The following table contains an exemplary sequence of 

notations for a "prerequisites review" education paradigm, and 
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the parameters relevanr m 

relevant to agent action processino th^ 
notation sends to the agent. 1,9 that eacn 

REVIEW 



Instructional r 
Sequence Entry UnTt^ame^L, Prere ^isite S Review 

10 S VS 5 a9e Time: 5 minutes 

Number of Exercises.- 8 

Exercise l Entrv ^ 

Header ***** £°™ a t Type: Multiple Choice 

Number of Options.- 2 n ° lce 
Difficulty: 0.3 
Hint: Available 



15 



Exercise k Entry „ 

Header y JSSKLIJ? 8 -- Fil1 in Blank 

Subformat: Select from Words in 

Difficulty:^ 0115 



An aspect of these notations is to provide informs ■ k 
t ne quality of the student response which In tTIsT 
- given by the a priori probability of a corre t a 
For example, selecting the correct Ln ^ 
25 has an a p.ior probability 7 " 
25 random. Filii na in * m \ ° B 13 made at 

instructional format can ba stored as IZ,, r ^ 
Notations are avaUabXe in ^Z^T^'^ 
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The standardization of the student data object according 
to this exemplary embodiment of IMIS for elementary education 
is simply achieved by storing student performance data 
according to instructional context, instructional format, and 
5 subject area. Thereby, these characteristics can be taken 
into account when comparing student pedagogic performance in 
general across subject matter areas. 

The final component of IMIS standardization is that of 
the agent action processing tables, the policy filter table, 

10 the decision weight table, and the selection rules. Event 
messages from the materials inform the agent of current 
values for the instructional context and instructional 
format. Since these values are parameters available to 
evaluate the conditions and functions contained in these 

15 tables, these tables can be, in effect, segmented into parts 
each corresponding to a particular instructional context and 
instructional format. Since there is considerable overlap 
between the parts of these tables, the number of rules does 
not proliferate. The current subject area is also available 

20 to segment the tables in the cases of those subjects that can 
require special treatment by agent action processing. 
Further, the IMIS standardization permits a more systematic 
and effective use of the mechanism which the agent uses to 
set global variables in the materials. These variables 

25 facilitate adaptive. adjustments of instructional parameters, 
such as seeding rate and amount of prompting. These 
variables can be more effectively set in view of the current 
educational paradigm as indicated by the current values of 
the instructional context and f ormat . IMIS is also useful in 

30 providing information to the student in response to "Where am 
I?" inquiries. The system can use the information contained 
in the entities in the information triple to respond, "You 
are halfway through the drill and practice on fractions," for 
example . 
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Alternative ng-enf embodim^ enabled h y 

In alternative embodiments of agent action processing 
the segmenting of the policy filter table and decision weight 
table, which provides a simplified 'intelligent instructional 
5 agent' for each kind of instructional context, instructional 
format, can be augmented by other techniques from artificial 
intelligence. Thus, the customized rules and functions 
contained in the action table can be augmented software 

10 TuTT\ WhiCh SXtend " ent ^ ""^ctad based on 
10 such technxques as production rules systems with rule 

propagation or neural nets. Such additional modules could 
for example, find complex patterns of student error 
unanticipated by the instructional designer. As additional 
arzzfxcial intelligence methods are incorporated into the 
15 agent software, new materials notations can be added to the 
notation repertoire. 



5 - 6 - 3 - Agent Ac tion Process ino 
Agent processing is divided into agent action 
20 processing, which determines on-screen agent display actions 
..in response to input events, and agent behavior processing 
which transforms display actions into displavs to the 
student. m a preferred embodiment, agent action processing 

25 H SVent driVen ' ^ a9ent bShavior P-cessing 

25 is table based. This and the succeeding section describe the 

preferred embodiments of these divisions of agent processing 

Fig. 8 illustrates in more detail agent action 

processing. This processing is activated when event messages 

801, representing either input events sent from the I/O 

30 handlers or educationally meaningful messages from the 

materials, are sent to the agent software, it transforms the 
input event messages into lists of display actions that 
actuate the agent display and also has the important side 
effects of updating student data object 806, as indicated by 

35 arrow 815, and of setting materials control parameters, as 
indicated by arrow 816. it is possible a particular input 
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event can generate all or none of these outputs and side 
effects . 

Agent action processing proceeds through three steps: 
event filtering, candidate action weighting, and display 
5 action selection. In common, each of these steps references 
rules in an associated table of rules. These rules include 
relations, functions, and procedures, all of which reference 
input parameters. Input parameters can be any appropriate 
data carried in the input event message and any appropriate 

10 data stored in student data object 806. In particular, as 
indicated by arrow 815, fields from the following subtypes in 
the student data model are referenced: the current lesson 
subtype, the materials specific subtypes, and the agent 
pedagogic student model subtype. In alternative embodiments 

15 where the agent interface and data is structured according to 
an IMIS standard, event input parameters include parameters 
characterizing the current educational paradigm adopted by 
the materials. These parameters can be used to select rules 
applicable only to this paradigm for reference and 

20 activation. 

Turning now to the steps of agent action processing, the 
first step is event filtering 807, referencing policy filter 
table 803. This step is described with reference to Table 7, 
an exemplary policy filter table. 



30 



35 
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TABLE 7 ; POLICY FILTER TABLE 





1! POLICY PTT.TM fi»»«*« 


Active 


Rule f 


v 


tp " p " ■■■ <Pn)/(type, subtype, x,. x,, xj 


Y 


(item^diff>LIM) , (congratulation. diff 2 ) 


Y 


(rate>f* average) , (congratulation. rate, k ) 


Y 


(exercise done) , (conqratulat ion, done, r, t ) 






N 


(latency > m) , (hint, constructed ) 


[ N 


( (error rate<k) * (WMA>MAX) , . (rp^^^, rri - ir ^~ 





10 



» The first row illustrates a generic policy filter rule, while 
subsequent rows illustrate exemplary filter rules The 
generic rule has a condition and a consequent. The condition 
is boolean expression, B(), of one or more conditions P 
each of which is a function of the available parameters . " 
Default parameters in a rule can be overwritten by parameters 
from the mput event. The consequent is a list of an agent 
action type, -type.- an agent action subtype, "subtype," and 
zero or more parameters, x,. The type is a major mode of 
agent behavior, for example, "congratulate the student," and 
the subtype modifies that behavior, for example, -because of 
the student's rate." The parameters provide additional 
information to be used in constructing an utterance, for 
example the time that the student took to complete an item or 
the name of the item. Each rule can be marked active or 
inactive, possibly as a result of selection during materials 
initialization or as a result of selection according to IMIS 
educational paradigm parameters. Processing a filter rule 
consists, first, in evaluating each active boolean, and 
second, for each such active and true expression, placing the 
consequent and its parameters into the list of candidal 
actions. The list of candidate actions is then passed on to 
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action weighing 808, the next processing step. Also in the 
student data model, the current lesson subtype and materials 
specific data subtype are updated, if necessary, with data 
from the input event message. For example, upon item 
5 completion, performance results need to be updated. 

Action weighting 808 references decision weight table 
804 and assigns a numeric weight to each action in the input 
candidate action list. This step is described with reference 
to Table B an exemplary decision weight table. 

10 



TABLE 8: DECISION WEIGHT TABLE 




DECISION WEIGHT TABLE | 


Active 


Rule | 


Y 


(type, subtype) , weight function (x 3 , x 2 , xj 






Y 


(congratulation, dif f ) , weight function j(Y a ,Y 2 ,Y 3 ) 


Y 


(congratulation, done), weight function k(Z I# Z,) | 



20 

The first row illustrates a generic policy filter rule, while 
subsequent rows illustrate exemplary filter rules. The 
generic rule has an index and a weight function. The index 
is a type and subtype pair r and the consequent is a weight 

25 function of zero or more parameters, x n , with a value between 
0 and 1. Each rule can also be marked active or inactive, 
possibly as a result of selection during materials 
initialization or as a result of selection according to IMIS 
educational paradigm parameters. Processing the list of 

30 candidate actions consists of, for each candidate action, 

selecting the weighting rules that have a type and a subtype 
matching that of the action and then assigning a relative 
priority between zero and one to the action as determined by 
applying the weight function to the available parameters. In 

35 this manner all the candidate actions on the candidate action 
list are assigned a numerical weight. The list of weighted 
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candidate actions is than passed on to action selection 809, 

the last processing step. 

Action selection 809 references selection criteria table 

5 " 1 ? 9 final ° f aCti ° nE fr ° m the candidate 

5 actxon list. This step is described ^ ^ 

9, an exemplary selection criteria table. 



10 



TABLE 9; SELECTION CRITERIA TABLE 




SELECTION CRITERIA TABLE | 


Active 




N 


(criterion) 


N 


(use all actions with weiqht > MIN) 


Y 


(use the most hiqhlv weighted action) 



20 



25 



The selection criteria table consists of a list of available 
methods for the candidate action selection process A 
selection criterion uses the confuted weight and perhaps 
other parameters to select one or .ore final actions from the 
caudate weighted actions. Hxemplary selection criteria are 

; ; Stnted ±n the laSt tWO — • it is anticipated that one 
criteria is marked active as a result of selection during 
materials initialization or as a result of selection 
according to mis educational paradigm parameters The 
act.ve criteria is applied to select zero or more final 
actions, some final actions are executed locally to provide 
control to the materials by setting control parameters, as 
30 «*«ated *y arrow 816. Other final actions can cause update 
of fields on the student data object, in particular in the 
agent pedagogic model subtype, as indicated by arrow 815 
The remaining final actions are display actions which are 
passed to agent behavior processing in display action list 
80-. The student data object is updated to record th- 
selected display actions. 



35 
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Action processing is illustrated with reference to the 
exemplary rules in Tables 7, 8, and 9 by considering the 
results of processing two input events, one of type "item 
done" with a "difficulty greater that LIM" and one of type 
5 "exercise done." Applying the exemplary filter rules from 
Table 7 results in two candidate actions, one being 
(congratulation, diff) and the other being (congratulations, 
done) . No other exemplary boolean expression evaluates to 
true. Next the exemplary weighting rules from Table 8 are 

10 simply applied by selecting and evaluating the correct 

weighting function. The resulting exemplary candidate action 
weights are taken to be 0.8 for (congratulation, diff) and 
0.5 for (congratulations, done). Finally, the one active 
exemplary selection rule from Table 9 is applied which 

15 selects the one final action (congratulation, diff) . 

The preferred embodiment here illustrated involved no 
rule propagation. In alternative embodiments more 
sophisticated artificial intelligence programming techniques 
could be employed as is apparent to those of skill in the 

20 art. For example, rule propagation and general production 
rule systems could be used to transform events to actions. 
Other applicable techniques could involve neural nets, 
predicate logic, and so forth. 

25 5.6.4. Agent Behavior Processing 

Agent behavior processing transforms display actions 
into output displays to the student. In a preferred 
embodiment, agent behavior processing selects from tables of 
possible display behaviors based on the input display action 

30 list and its parameters and on fields in the student data 
object describing student preferences and recent agent 
display behaviors. Further, in a preferred embodiment, agent 
behavior processing generates either display scripts which 
are sent to the I/O handler, or scripts with associated data 

35 snips of sound and animation that are embedded in applets. 
In alternative embodiments, intermediate scripts are not used 
and behavior processing generates I/O handler commands in 
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real time. This section describes agent behavior processing, 
first, with respect to the table data structures used, and 
second, with respect to the processing steps. 

Generally, agent behavior processing includes two steps. 
5 First, the message and its emotional affect, which is to be 
displayed or spoken by the on-screen agent, called herein the 
"utterance," is selected according to the input display 
actions. Second, the utterance and affect are sent to the 
preferred persona object where it is integrated into a 
10 complete display, including animation, graphics and audio. 
The display is preferably represented as a script which is 
then sent to the I/O handlers, m behavior processing, the 
selected affect in an important parameter. The affect 
further characterizes the intent of the utterance. For 
15 example, an utterance of a "congratulations" type is 
associated with a positive affect that can range from 
"happy," in cases of superior performance, to "encourage," in 
cases of improving but still less than average performance. 
The display behavior of the on-screen agent, preferably 
20 represented as one or more personae interacting with each 
other and the student, is strongly responsive to the affect 
parameter. As is known in the arts of film animation, 
infusing an animation with an affect or emotion gives 
resulting images a human and life-like quality. Such a 
2 5 quality is important so that the virtual tutor aspect of the 
ABI system engage the student in order to improve 
instructional results. Also, by storing recent affects 
selected in the student data objects, as well as student 
preferences for such parameters as affect intensity and 
30 presentation, agent behavior processing can present a 

coherent and life-like imaje agreeable to the student over 
one instructional session or over several instructional 
sessions . 

Affects are associated with and further characterize the 
35 intent of utterances . They are selected as part of utterance 
generation processing for use in visual display generation 
processing. Utterance generation processing depends on the 
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type, subtype, and parameters of the current event as well as 
on the student data object, which contains the agent student 
mcdel, student pedagogic parameters, and recently generated 
behaviors. Therefore, preferably, the selected affect and 
5 generated visual displays are responsive to the student 
pedagogic characteristics or, alternatively, the student 
cognitive style. 

Turning details of utterance generation, utterances and 
associated affect are generated by selecting options from a 

10 hierarchy of tables. This hierarchy, having typically many 
choices for each level, provides variety in agent behavior. 
At the highest level of this hierarchy are the tables of' 
utterance templates 908, which are indexed by action type and 
action subtype. Typically, there are many utterance 

15 templates and associated affect for any given action type and 
subtype, and indexing is according to additional parameters 
that occur in action lists and additional parameters of 
student characteristics from the student data object. For 
example, utterances and affects are further selected 

20 according to the educational paradigm employed, student grade 
level, and student preferences. 

At the next level of the utterance generation hierarchy 
are tables of named slots, which are parametrized phrases, or 
in other words, functions which use input parameters to 

25 return phrases. Utterance templates contain named slots as 
well as invariable words and sounds. When an utterance 
template is evaluated, slots with the specified name are 
selected from the slot table. Typically, there are' many 
slots of a given name, which are further indexed according to 

30 additional action list parameters and user characteristics 
from the student data object. When a slot is selected for 
use in the utterance template, slot parameters are passed 
from the utterance template. Alternative embodiments of this 
invention are adaptable to hierarchies with additional levels 

35 of utterance generation tables or additional types of tables 
at a given level in the hierarchy. 
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Exemplary Table 10 is a small segment of the utterance 
template table for an action type of "congratulations" and an 
action subtype of -performance" and appropriate for a test 
type paradigm. Table 10 illustrates that the student grade 
5 level, obtained from the student data object, is an 

additional indexing parameter for these utterance templates - 
Further, each illustrated template includes a slot named 
"performance- streak- slot," with parameters of grade level 
number of problems answered correctly, and total number of 
10 problems. 

TABLE 10 : EXEMPLARY UTTERANCE TEMPLATES 



15 



(congratulations, performance, grade-4 "You aor 
<performance- S treak- S lot(4. bright , ^g^L.- 



20 



(congratulations, performance, arade-8 ^vhTTT '■ ~" ~ 
( conarariii ^tinnr _ ' 



— ■ iiuuu. xvjul , co uai ; > . " ) 

(congratulations, performance, grade=12 "A stellar 

ss ??rs; h rsf ins <perf ™ >-^-*&'£ 1 iL**t . 



It is preferable that the utterance template table have a 
large number of utterance templates appropriate to a large 

25 number of educations situations. Since the preferable number 
of type and subtype combinations is at least approximately 
500, and since it is preferable to have 10 to 100 responsive 
utterances per pair, a preferable utterance template table 
has at least 25,000 entries, a less preferable table has at 

3(j least 5,000 entries and a more preferable table has at least 
50,000 entries. in an embodiment of the invention, less used 
utterances have fewer candidate responsive utterances without 
impairing the image of system spontaneity. Thus this 
invention is adaptable to utterance tables of i.ooo entries 
35 Exemplary Table li is a small segment of the slot table 

for slots named "perf ormance-streak-slot , - any of which could 
be employed in the previous templates. The illustrated slots 
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evaluate their input parameters to results in phrases. For 
example the last slot results in the phrase ,f 90 percent" if 
numright is 9 and total is 10. Further, slots can contain a 
condition which controls their applicability. In this 
5 example it is the fifth component of each entry and only the 
first slot has an non-null condition. 



TABLE 11: EXEMPLARY SLOT ENTRIES 



10 


(performance- streak- slot , 4-12, 
> 0 .8*<total>, "Nearly all") 


numright , 


total, 


<numright> 




(performance -streak -slot, 8-12, 
"<:numright> out of <total>") 


numright , 


total, 


nil , 




(performance- streak- slot , 8-12 , 
" 1 0 0 * <numright > / < t ot al > percent 


numright, 
") 


total, 


nil , 


15 'I 



Slots are used to create final, generated utterances with 
considerable variety. Therefore, it is preferable that the 
slot tables have at least the same number of entries as the 
utterance template table. A preferable slot has at least 
20 25,000 entries. A less preferable table has at least 5,000 
entries, and a more preferable table has at least 50,000 
entries . 

The final display of the preferred on-screen agent 
object is generated from Display Behavior Tables 904 with a 

25 similar hierarchical table data structure to that used in 
utterance generation. At the highest level are on-screen 
agent actions which contain a cast of one or more personae. 
At the next level are persona types which the student selects 
once, or at most once per session. At the next level, 

30 associated with each persona type is a library of display 
behaviors indexed by affect. Exemplary affect types include 
the following: sad, objective, pleased, happy, disappointed, 
announce, remind, encourage, reinforce, model, prompt, hint, 
joke, and tutor. Typically, each affect has many possible 

35 behaviors and these are further indexed, as for utterance 
generation, by parameters appearing in the action list and 
the student data object. The display behaviors are 
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structured as scripts containing named display objects. 
These named display objects can optionally involve voice, 
audio, graphics, or video displays, and they are contained in 
scripts which can optionally specify a timed animated display 
5 or a branching display, where the branches are dependent on 
student reactions to the display. At the lowest level in the 
preferred embodiment are the individual named display 
objects. As for slots, typically there are several 
parametrized instantiations of each named object. These 

10 instantiations are indexed according to the same parameters 
indexing the display behaviors and in turn use these" 
parameters to generate displays. Alternative embodiments of 
this invention are adaptable to hierarchies with additional 
levels of tables for display generation tables or additional 

15 types of tables at a given level in the hierarchy. 

Exemplary Table 12 is a small segment with personae 
types adapted to elementary education. 



TABLE 12: EXEMPLARY ON-SCREEN AGENT ACTIONS 



20 



A frog persona writing the congratulatory message on a tiny 
blackboard, then jumping up and down croaking ' Hurray' 



A robot persona creakily waving its arms and saying the 
congratulatory message. 



25 



An animal persona wearing a cheerleader's outfit jumping up 
and down, chanting the congratulatory message as a cheer. 



The cat says the general part of the encouragement message 
while pointing to his bird sidekick. The bird then flies 
across the screen while saying the specific part of the 
message . 



30 it is preferable that the on-screen agent have a richly 

varied and engaging behavior. The persona types preferably 
include recognizable popular figures as well as a rich 
variety of specially created figures. Persona type tables 
have a preferable selection of at least 100, and less 

35 preferably at least 50 persona types. The libraries of 
display behaviors preferably have many behaviors for each 
affect. Since the preferable number of affects is at least 
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50, and since it is preferable to have 50 to 100 responsive 
behaviors per affect, a preferable behavior libraries have at 
least 2,500 entries, and more preferably 5,000 entries, per 
persona type. A fully implemented ABI system can have 
5 preferable behavior libraries with approximately 125,000 
entries. This number can be less in situations were more 
popular persona have fully configured behavior libraries 
while less popular persona have more limited libraries. 
Further, the number of persona types can be advantageously 
10 limited to only the most popular or more appropriate for the 
type of student. Thus this invention is adaptable to 
libraries with approximately 1,000 entries of responsive 
behaviors . 

In implementations of this invention, it is important 

15 that the actual content of the utterance tables and the 
display tables and libraries be sufficiently creative and 
current to engage the student. Therefore, this content is 
preferably created by artists, animators, writers, and other 
creative talent. These elements of sound, voice, animation, 

20 graphics and video are collected into libraries of data snips 
and stored in archives. Further, it is preferable that these 
tables have an extensive and varied content in order that 
agent displays repeat only infrequently. 

Turning now to the process of generating a complete 

25 agent display behavior, the steps of this process are 
described with reference to Fig. 9. The first step is 
utterance generation 906 which receives as input action list 
901. Utterance generation indexes the utterance template 
table in accord with action list parameters, in particular 

30 action type and subtype, and student preferences from the 
student data object, to obtain candidate utterance templates 
responsive to the current situation. if no candidates are 
found, the student preferences are relaxed in a preset order 
until at least one candidate is found. Typically, more than 

35 one candidate is found, and then one template is selected at 
random from among those templates which have not yet been 
used previously in this session. A record of agent display 
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behavior during the session is stored in the student data 
object. The named slots of the selected template are 
resolved in a similar fashion, by finding candidate named 
slots and selecting one at random that has not been recently 
5 used. An utterance is then generated from the template, the 
named slot, and the input action list parameters. This 
utterance and its associated affect is passed to visual 
display generation. 

The second step of agent behavior processing, visual 

10 display generation 907, uses the input utterance and 

associated affect to select candidate responsive displays of 
that affect from the libraries associated with the student's 
current persona (e) preference contained in the student data 
object 905. As with utterance generation, if no candidate is 

15 found, student preferences are relaxed until one is found. 
Typically, more than one is found, and one is selected 
randomly avoiding recent selections in this session. Display 
objects referenced in the selected display are similarly 
found. Finally, the utterance message is sent to the 

20 selected display and display objects. These objects contain 
performance methods that are called to generate and output 
display script. In a preferred embodiment, the script with 
references to data snips 909 of voice, sound, graphics, video 
and animation is incorporated into an applet. An exemplary 

25 applet can be a program fragment representing a character 
scratching his head while saying the utterance and moving 
across the screen. This output applet is sent to the I/O 
handlers in the ES for student display 908. Since this 
completes agent processing for this student or timer event, 

30 the system then waits for the next action at wait point 902. 
A display action can also reference a preformatted 
animated sequence stored in the data snip library 909 in 
which voice, sound, graphics, animation and video have been 
already integrated. In this case, portions of utterance 

35 processing 906 and visual display processing 907 can be 
bypassed. 
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In alternative embodiments, selection of utterance or 
persona candidates from the available contents can be done 
other than randomly. For example, agent behavior data saved 
during a session in the student data object can be used to 
5 construct performances extending across instances of student 
actions. In particular, utterance templates and display 
objects can be selected in a connected manner in order to 
give agent display the appearance of continuity and realism. 
This invention is adaptable to other methods of generating 
10 performances parametrized by affect and utterance that are 
known in the art . These methods can include advanced text to 
speech techniques, 2-D and 3-D graphical generation, and' VR 
effects . 

The visualization of agent actions can involve a single 
15 character or several characters interacting with each other. 
If multiple characters are used, they are preferably 
interrelated. A typical cast is a cat with a bird sidekick. 
An exemplary interaction of these associated characters 
involving both dialogue and movement is presented in Table 
20 12. 

Continuing story line scripts can be available from the 
ABI system on a daily basis in a manner similar to a 
continuing plot in a daily comic strip. These story lines 
can be applicable to a group of students and individualized 

25 where appropriate. In some cases the student's selected 

persona can introduce a story line and in other cases several 
personae are directly involved in an interaction. For 
example, the story line can be used as the basis for a reward 
such as "We're off to the beach to play volleyball. Join us 

3 0 when you finish your homework, then later help us solve the 
mystery of the vanishing lake." 

In summary, the use of the student data object in agent 
behavior processing is illustrated in Table 13. 
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TABLE 13: STUDENT DATA USE IN AGENT BEHAVIOR 



5 



10 



15 



STUDENT DATA 


EFFECTS 


Student persona choice 


Display persona, including 
type of character, voice 
inflection, activity rate, 
and so forth 


Student s characteristics, 
including grade level, 
language ability, 
t*-*- ^ *• ciiL*c o , diiu so r or cn 


Narrow candidate utterance 
template and candidate 
display 


Previous student 
interactions with materials 
cr agent 


In alternative embodiments, 
utterance and display can 
relate to previous 
interactions 


Previous utterances in this 
session 


Selection of different 
utterance template and slots 


Previous displays in this 
session 


Selection of different 
display and display objects 



Particular data in the student data object has an important 
role in determining agent display and insuring its 
20 appropriateness, variety, and individualization. 



5-6.5. Aoent Adaptivitv 
The adaptation of the agent to the student, and the ABI 
system's appearance as a virtual tutor, emerges from the 

25 agent's updating of data in the student data object. In the 
preferred embodiment, the agent's knowledge of the student is 
represented by data in the student data object, in particular 
in the student pedagogic model, which stores general 
materials independent data describing how the student learns, 

30 and in the materials specific student performance, which 
stores performance data specific to particular assigned 
materials and courses. As the student interacts with the 
system for instruction or homework, the agent receives event 
messages, which describe the student's learning and 

35 performance. The agent updates the student data object with 
data from these messages. As this data is updated, the agent 
adapts to the student, and thereby the virtual tutor 
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individualizes to the student. This adaptation is maintained 
across sessions with this student. 

In the preferred embodiment, the data referenced and 
updated by the agent are averages or weighted moving 
5 averages, giving more weight to recent than past behavior. 
Thus, the pedagogic model includes, for example, data 
weighted moving averages of the rates that the student learns 
discrimination of a certain complexity. Materials specific 
performance includes, for example, weighted moving averages 

10 of data on the student's response time and response latency. 

In alternative implementations, agent adaptivity to its 
student can occur differently. In one alternative 
embodiment, student model data includes not only averages or 
weighted moving averages, but also data on the statistical 

15 distribution of the parameters involved, such as standard 
deviations. With this data the agent can recognize the 
current situation as "normal" or "abnormal" for this student 
and thereby offer appropriate guidance. Further, this 
statistical data can optionally include correlations between 

20 the agent data, such as between various pedagogic parameters 
and various materials parameters. In this manner, 
educational situations can be classified more finely than 
"normal" or "abnormal" into, for example, "abnormally slow on 
this fluency drill in view of normal progress on other 

25 exercises." In a further embodiment, classification of this 
statistical data can be done by special executable modules in 
the agent based on, for example, statistical classification 
schemes or neural networks . 

In a further alternative embodiment, agent action 

30 processing can be implemented with more complicated 
techniques. In the preferred embodiment, agent action 
processing is done by rules without rule propagation. In 
alternative embodiments, rule propagation and full rule based 
systems can be used to transform events into actions. This 

35 invention is applicable to other techniques from artificial 
intelligence that can make this transformation such as 
Bayesian belief networks. 
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In another alternative embodiment, the student data 
object has data modeling student interests and preferences. 
Such a model enables the agent, for example, to monitor 
school events and suggest those appropriate to the user. 
5 This model also enables the agent to provide rewards tailored 
to individual students, which enhances the system 
reinforcement and adds to perceived agent persona personality 
and to virtual tutor individualization. in a preferred 
embodiment, this model of student interests can be 

10 implemented simply as a set of approximately 200 categories, 
covering interest in several subdivisions of each school 
subject area, as well as categories related to sports, 
leisure time and other areas of student interest. Interest 
in these categories can be entered in several manners. 

15 Student use of the encyclopedia tool can be used to determine 
areas of current interest. Interest can be directly entered 
by the student, parent, or teacher. Alternately, student 
interest in materials can inquired for when the materials 
terminate. After an assignment, the student could provide 

20 semiotic feedback by selecting from a row of faces with 
different expressions. Alternately, the student can be 
quizzed on interests in a posed branching manner. 

6 • SPECIFIC EMBODIME NTS. CITATION OF REFERENCES 

25 The present invention is not to be limited in scope by 

the specific embodiments described herein. Indeed, various 
modifications of the invention in addition to those described 
herein will become apparent to those skilled in the art from 
the foregoing description and accompanying figures. Such 

30 modifications are intended to fall within the scope of the 
appended claims . 

Various publications are cited herein, the disclosures 
of which are incorporated by reference in their entireties. 

35 
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WHAT IS CLAIMED IS: 

1. A method of operating an agent based instruction 
5 system for interactive instruction of a student over a 

plurality of instructional sessions, said method comprising: 

(a) presenting interactive instruction to said 
student by executing one or more materials on a computer 
accessed by said student for a current instructional session,* 
10 (b) monitoring said interactive instruction of 

said student during said current instructional session; 

(c) storing information responsive to said 
monitoring of said student during said current instructional 
session for use during subsequent instructional sessions; and 
15 (d) outputting information on said computer to 

guide said student in said interactive instruction, said 
output information responsive to said monitoring of said 
student during said current instructional session and to said 
stored information responsive to said monitoring of said 
20 student during previous instructional sessions; 

whereby said system acts as a virtual tutor adapted 
to said student, said virtual tutor for guiding said 
interactive instruction of said student. 

25 2. The method according to claim 1 further comprising 

after said step of monitoring a further step of controlling 
said one or more materials, said controlling responsive to 
said monitoring of said student during said current 
instructional session and to said stored information 

3 0 responsive to said monitoring of said student during previous 
instructional sessions, and wherein said one or more 
instructional materials are responsive to said controlling,- 
whereby said interactive instruction presented by 
said system is individualized to said student. 

35 

3. The method according to claim 2 wherein said step 
of monitoring further comprises monitoring pedagogic 
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characteristics of said interactive instruction of said 
student by said one or more materials, and said step of 
controlling further comprises controlling said one or more 
materials in order to present interactive instruction with 
5 said pedagogic characteristics. 

4. The method according to claim 3 wherein said 

pedagogic characteristics are selected from the group 

comprising time pacing of interactive instruction, new 

10 concept seeding rate, density of examples, and discrimination 
difficulty. 

5 . The method according to claim 2 wherein said step 
of monitoring further comprises monitoring according to an 

15 instructional context and an instructional format adopted by 
each of said one or more materials. 

6. The method according to claim 2 further comprising 
before said step of controlling and said step of outputting a 

20 further step of generating at least one action responsive to 
said monitoring of said student during said current 
instructional session and to said stored information 
responsive to said monitoring of said student during previous 
instructional sessions, and wherein said step of controlling 

25 is responsive to said generated action (s) and wherein said 
step of outputting is responsive to said generated action (s). 

7. The method according to claim 6 wherein said of 
generating step generates action {s) according to one or more 

30 tables of rules. 

8. The method according to claim 6 wherein said of 

generating step generates action (s) according to one or more 

methods selected from the group of expert systems, neural 

35 networks, Bayesian belief networks, and statistical pattern 
recognition. 
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9. The method according to claim l wherein said step 
of outputting information further comprises (i) a step of 
selecting an utterance and an affect from one or more 
utterance tables in a manner responsive to said monitoring of 
5 said student during said current instructional session and to 
said stored information responsive to said monitoring of said 
student during previous instructional sessions, and (ii) a 
step of selecting a visual display from one or more tables of 
display behaviors in a manner responsive to said utterance, 
10 said affect, and said monitoring and said stored information. 

10. The method according to claim 9 wherein said step 
of monitoring further comprises monitoring pedagogic 
information describing the pedagogic characteristics of said 

15 student, wherein said selected display behavior comprises one 
or more personae, and wherein said selected visual display is 
further responsive to said pedagogic information,- 

whereby said one or more personae present a life- 
like appearance individualized to said student's cognitive 

20 style. 

11. The method according to claim l further comprising 
prior to said step of outputting a step of inputting at least 
one request of said student for guidance in said interactive 

25 instruction, and wherein said output information is further 
responsive to said input requests. 

12. The method according to claim l wherein said 
computer accessed by said student is one computer of a 

30 plurality of computers interconnected by a network, and 
wherein said student cai, access any of said plurality of 
computers for an instructional session. 

13. The method according to claim l wherein said output 
35 information further comprises an utterance,, and wherein the 

step of outputting further comprises a step of selecting said 



- 123 - 



WO 97/44767 



PCT/US97/08687 



utterance from one or more candidate utterances, and a 
further step of outputting said utterance as text or speech. 

14 . The method according to claim 13 wherein said step 
5 o, selecting further comprises selecting said one or more 

candidate utterance from one or more tables of utterances. - 

15 . The method according to claim 14 wherein said one 
or more table of utterances comprise at least i.ooo 

10 utterances. 

16. The method according to claim 13 wherein said one 
or more candidate utterances comprises at least 10 
utterances . 



15 



17. The method according to claim 1 wherein said 
responsive output information further comprises a display 
behavior, and wherein the step of outputting further 
comprises a step of selecting said display behavior from one 

20 or more candidate display behaviors, and a further step of 
outputting said display behavior. 

18. The method according to claim 17 wherein said step 
of selecting further comprises selecting one or more 

25 candidate display behaviors from one or more tables of 
display behaviors. 

19. The method according to claim 18 wherein said one 
or more tables of display behaviors comprise at least 1,000 

30 display behaviors. 

20. The method according to claim 17 wherein said one 
or more candidate display behaviors comprises at least 10 
display behaviors. 

35 

21. The method according to claim 17 wherein said step 
of outputting said selected display behavior comprises 
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output ting one or more modalities selected from the group 
consisting of text, voice, audio, animation, video, and 
Preformatted animated sequences. 

5 22. The method according to claim 17 wherein said 

selected display behavior comprises one or more persona. 

23 . The method according to claim 1 wherein said one or 
more materials is a plurality of materials. 

10 

24. The method according to claim 1 wherein said one or 
more students is a plurality of students. 

25. The method according to claim l wherein said step 
15 of monitoring further comprises monitoring pedagogic 

information describing the pedagogic characteristics of said 
student in a manner independent of the subject matters of 
said one or more materials. 

20 26 * The method according to claim 1 wherein said step 

of monitoring further comprises monitoring progress and 
performance information describing the progress and 
performance of said student in said interactive instruction 
presented by each of said one or more materials. 

25 

27. The method according to claim 1 wherein said step 
of storing further comprises storing said information 
responsive to said monitoring in one of one or more student 
data objects, each student data object of said one or more 

30 student data objects being uniquely associated with exactly 
one student of said one or more students . 

28. The method according to claim 1 wherein said one or 
more materials further comprise a material engine and one or 

35 more materials data, and wherein said step of presenting 
interactive instruction by executing one or more materials 
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references said one or more material data. 

29. The method according to claim 26 therein said 

T-171TIT £urcher compriEe " otaci °- - »-- S aid 

s^p of momtonng further comprises said materials engine 

said method comprising: 

(a) checking the authority of one of »ih «... 
to access said svstem £or interactive' instruc t o y for 
15 mere materia at , computer, said computer being one 

:::: r ° £ a piuramy ° f ^r^c e : ; . 



said inn "" l0adin9 " Said CC "P ut « software and data for 
said interactive instruction, 

" m . eMCU " n 3 said °™ °r more materials on said 

s c t°r:: ~ n eo 

(d) outputting information on said computer to 
guide said student in said interactive instruction said 
25 output information responsive to performance of said 
interactive instruction of said student. 

after said^ "Tf ^ Claim 3 ° fUrth ^' -prising 

30 llll I P ° adin9 3 fUrthSr Ste P of monitoring said 

30 xnteractxve instruction, said monitoring generating 

monitoring information that monitors said interactive 
instruction of said student, and wherein said step of 

Infol? 9 inf ° matiOR iS »"P«.iv to said monitoring 
information; 3 

35 whereby said system acts as a virtual mr^ * 

«o said one student, said virtual tutor f d n a " 

interactive instruction of said student. 
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32. The method according to claim 31 further comprising 
after said monitoring step a further step of controlling said 
one or more materials to present instruction to said student 
that is responsive to said monitoring information, whereby 

5 said interactive instruction presented by said system is 
individualized to said student. 

33. The method according to claim 32 further comprising 
after said loading step a further step of maintaining a model 

10 of said one student, said maintaining responsive to said 

monitoring information of said student, and wherein the step 
of outputting and the step of controlling are further 
responsive to said model of said student, whereby said 
outputting step and said controlling step adapt to said 

15 student thereby individualizing said interactive instruction 
of said student. 

34 . The method according to claim 30 wherein said one 
or more materials comprise a plurality of materials. 

20 

35. The method according to claim 30 wherein one or 
more of said plurality of computers are configured as one or 
more server computers for holding databases of software and 
data, and said step of loading further comprises loading 

25 across said network to said computer from said databases. 

36. The method according to claim 35 wherein said 
loading occurs when said software and data is demanded at 
said computer . 

30 

37. The method according to claim 35 further comprising 
caching read-only data on said computer. 

38. The method according to claim 30 wherein said step 
35 of outputting information further comprises outputting 

information in one or more output modalities. 
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10 



15 



39. The method according to claim 38 where said one or 
more output modalities are one or more output modalities 
selected from the group consisting of text, graphics, speech 
aucxo, animation, video, and Preformatted animated sequences'. 

40. The method according to claim 38 wherein said step 
of outputting seIe cts said output modalities to output one or 
more persona or personae responsive to said interactive 
instruction. 

41. The method according to claim 38 wherein said 
output modalities further comprise content, and said step of 
outputtmg information further comprises a step of loading 
said content to said computer. 



42. A method of operating an agent based instruction 
system for interactive instruction of one or more students 
over a plurality of instructional sessions, said method 
comprising: 

20 <a) P rese "tin9 interactive instruction to one of 

said students by executing one or more materials on a 

s C "sion r aCC6SSed ^ StUd6nt 3 CUrrent ^""ctional 

, c <b) monit °ring sa id interactive instruction of 

25 saxd student during said current instructional session; 

(c) storing information responsive to said 
monitoring of said student during said current instructional 
session for use during subsequent instructional sessions.- 

(d) determining an affect responsive to said 

30 monitoring of sai d student during said current instructional 
ses S1 on and to said stored information responsive to said 
monitoring of said student during previous instructional 
sessions; and 

_ B (6) out P utc in<J information on said computer to 

.5 guide said student in said interactive instruction, said 
output information responsive to said affect, to said 
monitoring of said student during said current instructional 
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session, and to said stored information responsive to said 
monitoring of said student during previous instructional 
sessions, said output information comprising a visual 
display; 

5 whereby said visual display is responsive to said 

affect and to said interactive instruction in a life-like 
manner . 

43. The method according to claim 42 wherein said step 
10 of monitoring further comprises monitoring pedagogic 

characteristics of said student, and wherein the step of 
determining an affect determines an affect further responsive 
to said pedagogic characteristics ; 

whereby said visual display is responsive in a 
15 life-like manner individualized to said student's cognitive 
style . 

44. The method according to claim 42 wherein said 
output information further comprises an utterance, and 

20 wherein the step of outputting further comprises a step of 
selecting said utterance from one or more tables of 
utterances in a manner responsive to said monitoring and to 
said stored information, and a further step of outputting 
said utterance as text or speech; 

25 whereby said utterance and said visual display are 

responsive in a life-like manner. 

45. The method according to claim 42 wherein said step 
of outputting further comprises selecting said visual display 

30 from one or more tables of visual displays in a manner 

responsive to said affect, to said monitoring, and to said 
stored information. 

46. The method according to claim 42 wherein said 
35 selected display behavior comprises one or more persona. 
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instruction system £or interactive 

instruction of on. or more students, said system comprising- 

im ., °" e ° r more c <^«re having interactive 

input/output devices and interconnected by a network.. 

or „„ ^ ° r "°™ materials «ecutable on said one 

or .ore computers, each said material for presenting 
interactive instruction to said one or more students and for 
generating monitoring information that monitors said 
interactive instruction; and 

10 ,«,, <Cl ^ ° r °°" a9en " on said one or 

7' each said agent associated with exactly Z 

Of said students and each said student associated w" 

exactly one of said agents, each said agent comprising 

,, „„ <l) aCCl ° n Passing for controlling said 

said" ™ terialS C ° ±nStrUCt ~ 

sa d controlling being responsive to said monitoring 

information that monitors said interactive instruction of 
said associated student, and 

20 ^ . Ui) behavior Processing for outputting 

20 Ration to guide said associated . ^ J 

being responsive to said Storing information that m Litor S 
said interactive instruction of said associated student- 

whereby said system acts as a virtual tutor to each 

25 III T UdentS ^ instruction of each 

25 said students is individualized to each student. 

48. The system according to claim 47 further comp ris i na 
executive software for interfacing said _ ^ ™ 

30 ?JT£Z m ~ t0 S - ~ « - — rs at 

49. The system according to claim 47 wherein said 
network is configured to permit any one of said students to 
access any one of said one or more materials from any one of 

35 said computers. 
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50. The system according to claim 47 wherein said 
network is configured to permit one or more of said computers 
to be located in one or more locations. 

5 51. The system according to claim 50 wherein one or 

more of said locations are residences of one or more of said 
students . 

52 . The system according to claim 50 wherein one or 
10 more of said materials presents homework to one or more of 

said students. 

53. The system according to claim 47 wherein said 
network is further configured to be a packet switched 

15 network. 

54. The system according to claim 47 wherein said agent 
action processing is further responsive to a request for 
guidance from said associated student, and wherein said agent 

20 behavior processing is further responsive to said request for 
guidance . 

55. The system according to claim 54 wherein said agent 
behavior processing further comprises (i) utterance 

25 generation processing for selecting an utterance and an 

affect responsive to said monitoring information or to said 
request for guidance, and (ii) visual display generation for 
selecting a visual display responsive to said utterance and 
said affect. 

30 

56. The system according to claim 55 wherein said agent 
behavior processing selects said utterance and said visual 
display to represent a persona, said selection being from a 
table of available personas and according to the preferences 

35 of said associated student. 
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57. The system according to claim 47 wherein said 
material, further comprise instructional materials executable 
on saxd computers for interactive instruction of said 
students, and tools executable on said computers for 

5 assistance of said students in said interactive instruction 

58. The system according to claim 57 wherein said 
instructional materials further comprise instructional 
materials appropriate to primary or secondary education. 

furrh The . SyStem acc ° rdi *9 t:o claim 57 wherein said tools 
further comprxse one or more tools selected from the group 
comprxsxng a calculator, a dictionary, a thesaurus, an atlas 
an encyclopedia, and a network search tool. 

furth 60 • Th6 . SyStem according to claim 57 wherein said tools 
further comprxse a starfish to6l for displaying amJ sele 

relationships. , y 

20 61. The system according to claim 47 further comprising 

one or more schedule/calendar tools executable on said 
computers, each said schedule/calendar tool associated with 
exactly one of said plurality of students and each said 

25 t'oolT aS h S ° Ciated With «■ of said schedule/calendar 

25 tools, each saxd schedule/calendar tools for outputting to 
saxd associated student information relating to scheduled 
actxvxtxes of said associated student, and for providing to 
saxd agent of said associated student information relating to 
scheduled activities of said associated student 

30 

62. The system according to claim 61 further comprising 
data areas for each of said students characterizing scheduled 
actxvxtxes according to deadline date and priority, and 
wherein said schedule/calendar tool further limits said 
3 5 associated student to interactive instruction according to 
saxd data areas characterizing said scheduled activities 



132 - 



WO 97/44767 PCT/US97/08687 



63. The system according to claim 47 further comprising 
one or more communication tools executable on said computers 
for providing forms of interactive group instruction to a 
group of said students, and for generating monitoring 

5 information that monitors said group instruction of each 
student in said group. 

64 . The system according to claim 63 where said forms 
of group instruction are selected from the group consisting 

10 of exchange of messages, group work on a shared material, and 
group participation in educational contests. 

65. The system according to claim 47 wherein one or 
more of said materials further comprises materials engine 

15 software and materials data, and wherein said materials 
engine process said materials data to present said 
interactive instruction . 



66. The system according to claim 65 wherein each of 
20 said materials data further comprises: 

(a) a plurality of display objects for 
presentation; 

(b) sequencing logic for controlling the order of 
said presentation of said plurality of display objects; and 

25 (c) notations for causing generation of said 

monitoring information. 

67. The system according to claim 47 wherein one or 
more of said materials comprises a program having data and 

30 instructions. 



68. The system according to claim 47 further comprising 
pedagogic information data areas for each student, said 
pedagogic information data areas comprising data for a 
35 pedagogic model of said student, and wherein said agent 
action processing and said agent behavior processing is 
further responsive to said pedagogic information data areas. 
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in sai f' T SyStem aCC ° rdin9 t0 clai ™ 68 wherein said data 
xn saxd pedagogxc information data areas does not depend on 
the subject matter of said one or more materials. 

> 70. The system according to cl aim 68 Mherein said 

actxon processing updates said pedagogic information data 

up r d e a a t S in a r b °e Ciated ^ ^ * S ~ 

updatxng bexng responsive to said monitoring infection, 

whereby saxd associated agent adapts to said student. 

71. The system according to claim 68 further comprising 
progress and performance data areas for each student s2T 
Progress and performance data areas comprising data 
descrxbxng progress and performance of eac h said student in 
15 saxd xnteractive instruction, and wherein said agent action 
processxng and said agent behavior processing arl further 
responsxve to said progress and performance information data 

20 ap ,. 72 • ^ SyStem acc °^ng to claim 7i wherein said agent 
actxon processing updates said progress and performance J^' 
are as assocxated with said associated student of said agent 
saxd updatxng responsive to said monitoring information! 

25 student. ^ aSS ° Ciated a9Snt adaptS t0 Said 

73. The system according to claim 47 wherein one or 
-ore of said materials presents said interactive instruction 
accordxng to an education paradigm and wherein said 
30 monxtoring information generated by said materials further 
comp: xses pedagogic information classified according to said 
education paradigm of said materials. 

35 conr T " ^ aCC ° rding t0 clain > ^ further comprising 

ZrlTST teiS material ^ " education^ 9 

paradxgm, wherexn each material presents said interactive 

instruction according to values of said control parameters. 
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and wherein said agent action processing assigns said values 
of said control parameters; 

whereby said agent action processing controls said 

materials . 

5 

75. The system according to claim 73 wherein said 
educational paradigm is selected from the group consisting of 
interactive tutorial, fluency exercise, paired association 
exercise, discrimination formation exercise, and simulation 

10 exercise. 

76. The system according to claim 73 wherein said 
educational paradigm is standardized according to an 
instructional context and an instructional format. 

15 

77. The system according to claim 76 wherein said 
instructional context is selected from the group consisting 
of prerequisite test, prerequisite review, pretest, new 
material introduction, new material discrimination, new 

20 material review, fluency exercise practice, review practice, 
and unit mastery test . 

78. The system according to claim 76 wherein said 
instructional format is selected from the group consisting of 

25 multiple choice, unprompted fill-in- the-blank, fill-in-the- 
blank selected from a list, paired associates by letter, 
paired associates by dragging, paired associates by matching, 
computation, simulation to identify parts of figures or text, 
and simulation game. 

30 

79. The system according to claim 47 wherein said agent 
action processing generates at least one action responsive to 
said monitoring information. 

35 80. The system according to claim 79, wherein said agent 

action processing further comprises software using one or 
more techniques selected from the group of expert systems, 
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neural networks. Bayesian belief networks, and statistical 
pattern recognition. 

81. The system according to claim 79 further comprising 
5 a plurality of tables of rules, and wherein said agent action 
processing software further comprises software referencing - 
said plurality of tables of rules in order to control said 
one or more materials and to generate said action (s) 



10 



82 . The system according to claim 81 wherein said 
Plurality of tables of rules comprises a policy filter table 
a decision weight table, and a selection criteria table and' 
wherein said agent action processing references said policy 
filter table for determining one or more candidate actions 
15 references said decision weight table for ranking said one 'or 
more candidate actions, and references said selection 
criteria table for selecting said action ( S ) from said ranked 
candidate actions. 

20 83. The system according to claim 79 wherein said agent 

behavior processing is responsive to said action (s) in order 
to output information responsive to said monitoring 
information. 



25 84. The system according to claim 83 further comprising 

a plurality of tables of possible outputs, and wherein said 
agent behavior processing software references said tables of 
possible outputs . 

30 85. The system according to claim 84 wherein said one 

or more tables of possible outputs further comprise an 
utterance template table and a display behavior table, and 
wherein said agent behavior processing references said 
utterance template table for selecting an utterance and an 

35 affect according to said action(s) , and references said 
display behavior table for selecting a visual display 
according to said action(s), said utterance, and said affect 
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86. The system according to claim 83 wherein said agent 
behavior processing outputs information in a plurality of 
output modalities . 

5 87. The system according to claim 83 wherein said 

output modalities are one or more output modalities selected 
from the group consisting of text, graphics, speech, audio, 
animation, video, and preformatted animated sequences. 

10 88. The system according to claim 47 further comprising 

one or more student data objects, each of said student data 
objects associated with exactly one student, and wherein each 
agent stores information reflecting said monitoring 
information of said one student associated with said agent in 

15 said one student data object associated with said associated 
student . 

89. The system according to claim 88 wherein said 
student data object associated with one student further 

20 comprises: 

(a) pedagogic information describing a pedagogic 
model of said one student; and 

(b) progress and performance information for 
describing the progress and performance of said student in 

25 said materials. 

90. The system according to claim 89 wherein said agent 
associated with each student further comprises agent software 
and said student data object associated with said student, 

3 0 and wherein said agent software references and updates said 
associated student data object. 

91. The system according to claim 47 wherein one or 
more of said computers are configured as server systems, 

35 wherein said server systems store said one or more materials 
and one or more agents, and wherein said one or more 
computers downloads said materials and said one or more 
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agents from said one or more server systems across said 

computer network. 

92. The system according to claim 91 further comprising 
5 databases of pedagogic information and materials progress and 

performance information for said one or more students, and 
wherein said databases are stored on said server systems. 

93. The system according to claim 92 further comprising 
10 reporting software for generating reports from said databases 

of pedagogic information and materials progress and 
performance information for said one or more students. 

94. The system according to claim 47 further comprising 
15 one or more system managers executable on said one or more 

computers, wherein said interactive input/output devices 
include a computer display at each of said one or more 
computers, and wherein said system manager executing on one 
computer partitions said display of said one computer into 
20 one or more display areas. 

95. The system according to claim 94 wherein said one 
or more display areas are selected from the group comprising 
an agent area for agent interactivity, a materials area for 

25 materials interactivity, and a system area in which said 

system manager displays selection icons for available system 
tools and facilities. 

96. The system according to claim 95 wherein said one 
30 or more materials and said one or more agents provide 

facilities always available to said o.ie or more students and 
Said Mt «i«l» ^a and said agent area comprise sub-areas 
whose selection activates one of said always available 
facilities . 



35 



97. The system according to claim 95 wherein said one 
or more display areas change in size from time to time. 
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98. The system according to claim 47 wherein one or 
more teachers directs instruction of one or more of said 
students, said system further comprising.- 

(a) one or more teacher materials executable on 
5 said one or more computers, each said teacher material for 
presenting interactive instruction to teachers in the use of 
said system and in the use of materials directed to said 
students and for generating monitoring information that 
monitors said interactive instruction; and 
10 (b) one or more teacher agents executable on said 

one or more computers, each said teacher agent associated 
with exactly one of said teachers and each said teacher 
associated with exactly one of said agents, each said agent 
comprising 

15 (i) action processing for controlling said 

one or more materials to instruct said associated teacher, 
said controlling being responsive to said monitoring 
information that monitors said interactive instruction of 
said associated teacher, and 

20 (ii) behavior processing for outputting 

information to guide said associated teacher, said outputting 
being responsive to said monitoring information that monitors 
said interactive instruction of said associated teacher. 

25 99. An agent based instruction system for interactive 

instruction of one or more students, said system comprising: 

(a) one or more computers having interactive 
input /output devices and interconnected by a network; 

(b) one or more materials executable on said one 
30 or more computers, each said material for presenting 

interactive instruction to said one or more students and for 
generating monitoring information that monitors said 
interactive instruction; and 

(c) one or more agents executable on said one or 
35 more computers, each said agent associated with exactly one 

of said students and each said student associated with 
exactly one or said agents, each said agent comprising 
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„. U) Utte ^nce generation processing for 

selectmgan affect and an utterance to guide s J d one 

^ ™ b : tudent ' said seiectin * ° f s - d — - -« 

5 that ZL "^^^ t0 Said -nitoring infection 

5 that monitors sa.d interactive instruction of said one 
associated student, and 

visual diml ViSUal diSPlay * enerati °n selecting a 

acc • (Ui) ° UtpUt Processing for outputting to said 

» :i;:\ ia :i: P ;ir c said seiected — - - 

selected ut^" S6leCted ViSUal d±Splay and 5ai * 

selected utterance are responsive to said affect and to said 

interactive instruction in a life-like manner. 

20 100. The system according to claim 99 wherein said 

-nitonng information further comprises pedagogic 
information describing the pedagogic characteristics of said 
assorted student, and wherein utterance generation 
processing selects an affect further responsive to said 

25 pedagogic information,- 

whereby said selected visual display and said 
selected utterance are further responsive in I J„ r 
individuated to a cognitive style of said associated 



30 



35 



101. The system according to claim 99 wherein sad 

TtllZll Pr ° CeSSin9 ^ -id 

utterance from one or more tables of utterances. 

102. The system according to claim 99 wherein said 
-sual display processing selects said visual display from 
one or more tables of visual displays. 
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103. The system according to claim 99 wherein said 
selected visual display comprises one or more persona. 

104. The system according to claim 99 further comprising 
5 action processing for generating at least one action and for 

controlling said one or more materials to instruct said 
associated student, said generating and said controlling 
being responsive to said monitoring information that monitors 
said interactive instruction of said associated student, and 
10 wherein said utterance generation processing and said visual 
display selection processing are further responsive to said 
action (s) . 

105. A method of operating an agent based instruction 
15 system for instruction of a plurality of students, said 

method comprising: 

(a) executing one or more materials on a computer 
for presenting interactive instruction to one student of said 
plurality of students, said computer being one computer of a 

20 plurality of computers interconnected by a network, each 
computer of said plurality having interactive input /output 
devices ; 

(b) generating monitoring information that 
monitors said interactive instruction presented to said 

2 5 student ; and 

(c) controlling said one or more materials to 
instruct said student, said controlling being responsive to 
said information monitoring the interactive instruction of 
said student; 

3 0 whereby said interactive instruction is 

individualized to each student of said plurality of students. 

106. The method according to claim 105 further 
comprising after said step of generating a further step of 

3 5 outputting information on said computer to. guide said student 
in said interactive instruction, said outputting information 
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responsive to said monitoring information of said interactive 
instruction of said student; overactive 

whereby said system acts as a virtual tutor adapted 
to said student, said virtual tutor for guiding said 
5 interactive instruction of said student. 

infill' T ^ meth ° d aCC ° rdin9 to clai * * wherein said output 
information further comprises one or more persona. 

10 108. The method according to claim 7 wherein said step 

of outputtmg information further comprises outputtihg 
information in a plurality of output modalities. 

109. The method according to claim 108 wherein said 
— "ti.. are one or more output ^d.^ 
from the group consisting of text, graphics, speech, aud 
animation, vxdeo, and preformatted animated sequences 

110 • The metn °<* according to claim ? wherein said step 
20 o outputtmg information further comprises (i, a step of P 
selecting an utterance- and an affect from one or more 

L^raTr ^ 3 ™ reSP ° nSiVe C ° Said strolling 

step and (u, a step of selecting a visual display from one 
or more tables of disp i ay behaviors in . J esponsi Z \ 0 

25 said utterance, said affect, and said controlling step. 

HI. The method according to claim ? further comprising 

IZIT ° UtPUttin9 St6P 3 fUrth6r ^ ° f inpu^in * 

student requests for guidance in said interactive 

30 is^tLT' ^ Wh6rein St6P ° f ° UtPUtti " 9 ^^-tion 

is further responsive to said student requests. 

112 . The method according to claim 105 wherein said one 
or more materials is a plurality of materials 

35 

113 . The method according to claim 105 further 
comprising prior to said executing step a further step of 
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accessing any one of said computers by said one student, said 
accessing comprising verifying the authority of said one 
student to access said system for interactive instruction by 
said one or more materials. 

5 

114 . The method according to claim 113 wherein one or 
more of said computers are located in a plurality of 
locations, and wherein the step of accessing further 
comprises accessing any of said computers in plurality of 

10 locations . 

115. The method according to claim 114 wherein one or 
more of said plurality of locations are residences of one or 
more of said students, and wherein said one or more materials 

15 presents homework in said one or more residences . 

116. The method according to claim 105 wherein said step 
of controlling further comprises controlling according to one 
or more tables of rules. 

20 

117. The method according to claim 116 wherein said one 
or more tables of rules further comprise a policy filter 
table, a decision weight table, and a selection criteria 
table, and wherein said controlling step controls said one or 

25 more materials according to one or more determined 

controlling actions, and wherein said controlling step 
references said policy filter table for determining one or 
more candidate controlling actions, references said decision 
weight table for ranking said one or more candidate 

30 controlling actions, and references said selection criteria 
table for selecting one or more controlling actions from said 
ranked candidate controlling actions. 

118. The method according to claim 105 wherein said step 
35 of controlling further comprises controlling according to one 

or more methods selected from the group consisting of expert 
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systems, neural networks, Bayesian belief networks, and 
statistical pattern recognition. 

119- The method according to claim 105 wherein the step 
5 of executing further comprises executing one or more of said 
one or more materials according to an educational paradigm 
and wherein said monitoring information further comprises ' 
pedagogic information classified according to said 
jj 6duCationai Paradigm adopted by said one or more materials. 

120. The method according to claim 119 wherein said 
educational paradigm is standardized according to an 
instructional context and an instructional format 



IS 



121 . The method according to claim 105 further 
comprising after said generating step a further step of 
updating one student model of a plurality of student models, 
each student of said plurality of students being associated 
with exactly one student model, said updating being 
20 responsive to the information monitoring the interactive 
instruction of said student, and wherein the step of 
controlling said one or more materials to instruct said 
student is further responsive to said student model of said 
student; 

25 whereby said one or more materials are 

individualized to said student. 

122. The method according to claim 121 wherein said step 
of updating further comprises updating pedagogic information 
30 m said student model, said pedagogic information describes 
pedagogic characteristics of said student of said plurality 
of students in a manner independent of the subject matters of 
said one or more materials . 

35 123. The method according to claim 121 wherein said step 

of updating further comprises updating progress and 
performance information in said student model, said progress 
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and performance information describes the progress and 
performance of said student in said interactive instruction 
presented by each of said one or more materials. 

5 124. The me: nod according to claim 105 wherein said one 

or more materials further comprises one or more instructional 
materials for presenting interactive instruction to said 
plurality of students, and one or more tools for presenting 
interactive assistance to said plurality of students during 
10 said interactive instruction. 

125. The method according to claim 105 further 
comprising prior to said controlling step (i) a step of 
executing one scheduler/calendar tools of a plurality of 

15 scheduler/calendar tools, each student of said plurality of 
students being associated with exactly one scheduler/calendar 
tool, said one scheduler/calendar tool providing said student 
with information relating to scheduled activities of said 
student, and (ii) a step of generating monitoring information 

20 that monitors said scheduled activities of said student. 

126. The method according to claim 125 wherein the step 
of executing executes one or more materials that relate to 
said scheduled activities of said student. 

25 

127. The method according to claim 105 further 
comprising prior to said controlling step (i) a step of 
executing one or more communication tools for presenting one 
or more forms of group instruction to a group of students, 

30 said group including said student, and (ii) a step of 

generating monitoring information that monitors said group 
instruction of said student . 

128. A method of operating an agent based instruction 
35 system for instruction of a plurality of students, said 

method comprising: 
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(a) a step for executing one or more materials on 
a computer for presenting interactive instruction to each 
student, said computer being one computer of a plurality of 

5 ZlllT h interC ° nneCted b ^ a -"■or*, each computer of said 
5 plurality having interactive input /output devices ,- 

(b) a step for generating monitoring information 
that monitors said interactive instruction presented to each 
student of said plurality of students; 

(C) a ste P for controlling said one or more 
10 materials to instruct each student, said controlling being 
responsive to said information monitoring the interactive 
instruction of each student; and 

(d) a step of outputting information to each 

15 TZT"^ 0rd6r ^ 9Uide SaCh StUd6nt in Said -teractive 
15 instruction, said outputting step responsive to said 

information monitoring the interactive instruction of each 

student and responsive to said controlling step,- 

whereby said interactive instruction is 

20 a " diVidUali26d t0 Sach Student ^ said plurality of students 
20 and acts as a virtual tutor individualized to each student. 

129. An agent based instruction system for instruction 
of a plurality of students, said system comprising: 

(a) means for presenting interactive instruction 
2 5 to each student by one or more materials , 

(b) means for generating monitoring information 
that monitors said interactive instruction presented to each 
student; 

(O means for controlling said one or more 
30 materials to instruct each student, said controlling being 
responsive to said information monitoring the interactive 
instruction of each student; and 

(d) means for outputting information to each 
student in order to guide each student in said interactive 
35 instruction, said means for outputting being responsive to 
sa ld information monitoring the interactive instruction of 
each student and responsive to said means for controlling. 
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whereby said interactive instruction is 
individualized to each student and said acts as a virtual 
tutor individualized to each student. 

5 130. An agent based instruction system for instruction 

of a student, said system comprising: 

(a) one or more materials executable on a computer 
for presenting interactive instruction to said student, said 
computer having interactive input /output devices; and 

10 (b) an agent executable on said computer, said 

agent (i) receiving monitoring information from each of said 
one or more materials that monitors said interactive 
instruction of said student, (ii) controlling said one or 
more materials to instruct said student, said controlling 

15 being responsive to said monitoring information, and (iii) 
outputting information to guide said student, said outputting 
being responsive to said monitoring information. 

131. A system for agent-based, interactive instruction 
20 of one or more students over a plurality of instructional 
sessions, said system comprising: 

(a) means for presenting interactive instruction 
to one of said students by executing one or more materials on 
a computer accessed by said student for a current 

25 instructional session; 

(b) means for monitoring said interactive 
instruction of said student during said current instructional 
session ; 

(c) means for storing information responsive to 
3 0 said monitoring of said student during said current 

instructional session for use during subsequent instructional 
sessions; and 

(d) means for outputting information on said 
computer to guide said student in said interactive 

35 instruction, said output information responsive to said 

monitoring of said student during said current instructional 
session and to said stored information responsive to said 
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monitoring of said student during previous instructional 

sessions; 

whereby said system acts as a virtual tutor adapted 
to said student, said virtual tutor for guiding said 
5 interactive instruction of said student. 

132. A system for agent-based, interactive instruction 
of one or more students over a plurality of instructional 
sessions, said system comprising: 

10 (a) means for Presenting interactive instruction 

to one of said students by executing one or more materials on 
a computer accessed by said student for a current 
instructional session; 

(b) means for monitoring said interactive 
15 instruction of said student during said current instructional 
session,- 



(O means for storing information responsive to 
said monitoring of said student during said current 
instructional session for use during subsequent instructional 
20 sessions; 

(d) means for determining an affect responsive to 
said monitoring of said student during said current 
instructional session and to said stored information 
responsive to said monitoring of said student during previous 

25 instructional sessions,- and 

(e) means for outputting information on said 
computer to guide said student in said interactive 
instruction, said output information responsive to said 
affect, to said monitoring of said student during said 

30 current instructional session, and to said stored information 
responsive to said monitoring of said student during previous 
instructional sessions, said output information comprisinq a 
visual display; 

whereby said visual display is responsive to said 
35 affect and to said interactive instruction in a life-like 
manner . 



- 148 - 



WO 97/44767 PCTAJS97/08687 



133. A computer readable medium comprising instructions 
for performing the method of claim 1. 

134. A computer readable medium comprising instructions 
5 for performing the method of claim 30. 

135. A computer readable medium comprising instructions 
for performing the method of claim 42. 

10 



15 



20 



25 



30 



35 
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